LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing
Open accessThe DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
Video creation has become increasingly popular, yet the expertise and effort required for editing often pose barriers to beginners. In this paper, we explore the integration of large language models (LLMs) into the video editing workflow to reduce these barriers. Our design vision is embodied in LAVE, a novel system that provides LLM-powered agent assistance and language-augmented editing features. LAVE automatically generates language descriptions for the user’s footage, serving as the foundation for enabling the LLM to process videos and assist in editing tasks. When the user provides editing objectives, the agent plans and executes relevant actions to fulfill them. Moreover, LAVE allows users to edit videos through either the agent or direct UI manipulation, providing flexibility and enabling manual refinement of agent actions. Our user study, which included eight participants ranging from novices to proficient editors, demonstrated LAVE’s effectiveness. The results also shed light on user perceptions of the proposed LLM-assisted editing paradigm and its impact on users’ creativity and sense of co-creation. Based on these findings, we propose design implications to inform the future development of agent-assisted content editing.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Human trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921 · cited 1×
“Our work builds upon existing literature in human-AI co-creation [8, 14, 21, 24, 30, 32, 39, 54, 56, 72] and further contributes by developing a new AI system for video editing and studying its impact.”
BackgroundGuidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · arXiv · doi:10.48550/arxiv.2304.03442 · Cited by 2,173
Large language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013
Large Language Models are Zero-Shot Reasoners
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · arXiv · doi:10.48550/arxiv.2205.11916 · Cited by 2,013
Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts
Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B., & Yang, Q. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581388 · Cited by 1,274
Reflexion: Language Agents with Verbal Reinforcement Learning
Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11366 · Cited by 1,087
Research works printed but not linked: 38
- 10.1080/19393555.2022.2060879 · DOI
- Stephen Brade, Bryan Wang, Mauricio Sousa, Sageev Oore, and Tovi Grossman. 2023. Promptify: Text-to-Image Generation through Interactive Prompt Exploration with Large Language Models. arxiv:2304.09337 [cs.HC] · arXiv
- Daniel Buschek, Lukas Mecke, Florian Lehmann, and Hai Dang. 2021. Nine potential pitfalls when designing human-ai co-creative systems. arXiv preprint arXiv:2104.00358 (2021). · arXiv
- 10.1145/3411764.3445131 · DOI
- 10.1145/3290605.3300931 · DOI
- 10.1038/s42256-020-0161-x · DOI
- Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David J. Fleet, and Tim Salimans. 2022. Imagen Video: High Definition Video Generation with Diffusion Models. arxiv:2210.02303 [cs.CV] · arXiv
- Wenlong Huang Pieter Abbeel Deepak Pathak and Igor Mordatch. 2022. Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents. arxiv:2201.07207 [cs.LG] · arXiv
- 10.1145/3290605.3300311 · DOI
- 10.1145/3544548.3581494 · DOI · Paper page
- 10.1093/jcmc/zmac014 · DOI · Paper page
- Ehud Karpas Omri Abend Yonatan Belinkov Barak Lenz Opher Lieber Nir Ratner Yoav Shoham Hofit Bata Yoav Levine Kevin Leyton-Brown Dor Muhlgay Noam Rozen Erez Schwartz Gal Shachaf Shai Shalev-Shwartz Amnon Shashua and Moshe Tenenholtz. 2022. MRKL Systems: A modular neuro-symbolic architecture that combines large language models external knowledge sources and discrete reasoning. arxiv:2205.00445 [cs.CL] · arXiv
- 10.1145/2470654.2481301 · DOI · Paper page
- 10.1145/3544548.3581006 · DOI
- Vivian Liu, Tao Long, Nathan Raw, and Lydia Chilton. 2023. Generative Disco: Text-to-Video Generation for Music Visualization. arXiv preprint arXiv:2304.08551 (2023). · arXiv
- 10.1145/3526113.3545621 · DOI · Paper page
- 10.1145/3563657.3596098 · DOI · Paper page
- 10.1145/3490099.3511159 · DOI · Paper page
- Federico Magni, Jiyoung Park, and Melody Manchi Chao. 2023. Humans as Creativity Gatekeepers: Are We Biased Against AI Creativity?Journal of Business and Psychology (2023), 1–14.
- Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Augustus Odena. 2021. Show Your Work: Scratchpads for Intermediate Computation with Language Models. arxiv:2112.00114 [cs.LG] · arXiv
- OpenAI. 2023. GPT-4 Technical Report. arxiv:2303.08774 [cs.CL] · arXiv
- 10.1145/3311957.3359433 · DOI
- 10.1145/3379337.3415864 · DOI · Paper page
- Jeba Rezwana and Mary Lou Maher. 2022. Identifying ethical issues in ai partners in human-ai co-creation. arXiv preprint arXiv:2204.07644 (2022). · arXiv
- 10.18653/v1/2020.emnlp-main.437 · DOI · Paper page
- 10.1145/3526113.3545617 · DOI · Paper page
- Derek Tam Anisha Mascarenhas Shiyue Zhang Sarah Kwan Mohit Bansal and Colin Raffel. 2022. Evaluating the Factual Consistency of Large Language Models Through Summarization. arxiv:2211.08412 [cs.CL] · DOI · Paper page
- 10.1145/2984511.2984569 · DOI · Paper page
- Maarten Van Someren, Yvonne F Barnard, and J Sandberg. 1994. The think aloud method: a practical approach to modelling cognitive. London: AcademicPress 11 (1994), 29–41.
- Pranav Narayanan Venkit, Sanjana Gautam, Ruchi Panchanadikar, Shomir Wilson, 2023. Nationality Bias in Text Generation. arXiv preprint arXiv:2302.02463 (2023). · arXiv
- 10.1145/3526113.3545680 · DOI
- 10.1145/3355089.3356520 · DOI
- Sitong Wang, Samia Menon, Tao Long, Keren Henderson, Dingzeyu Li, Kevin Crowston, Mark Hansen, Jeffrey V Nickerson, and Lydia B Chilton. 2023. ReelFramer: Co-creating News Reels on Social Media with Generative AI. arXiv preprint arXiv:2304.09653 (2023). · arXiv
- 10.1145/3379337.3415845 · DOI · Paper page
- 10.1145/3379337.3415882 · DOI · Paper page
- Zhengyuan Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. 2023. The dawn of lmms: Preliminary explorations with gpt-4v (ision). arXiv preprint arXiv:2309.17421 9, 1 (2023). · arXiv
- Yue Yu, Yuchen Zhuang, Jieyu Zhang, Yu Meng, Alexander Ratner, Ranjay Krishna, Jiaming Shen, and Chao Zhang. 2023. Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias. arXiv preprint arXiv:2306.15895 (2023). · arXiv
- 10.1145/3491102.3517479 · DOI · Paper page
Cited by
Zhang, S., Wang, H., & Yi, X. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757594 · Cited by 36
Amuse: Human-AI Collaborative Songwriting with Multimodal Inspirations
Kim, Y., Lee, S.-J., & Donahue, C. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713818 · Cited by 23
“Consequently, these systems are being applied across a growing range of creative pursuits, such as illustration [59], creative writing [18], and video editing [97].”
Data Playwright: Authoring Data Videos With Annotated Narration
Shen, L., Li, H., Wang, Y., Luo, T., Luo, Y., & Qu, H. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3477926 · Cited by 21
Cao, Y., Huang, Y., Truong, A., Shin, H. V., & Xia, H. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713401 · Cited by 16
Self-citation
Metadata
- Authors
- Wang, B., Li, Y., Lv, Z., Xia, H., Xu, Y., & Sodhi, R.
- Venue
- Proceedings of the 29th International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2024-03-18
- Pages
- 699-714
- OpenAlex
- W4391940619
- Cited by
- 63
- References
- 40
- FWCI
- 25.17
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing. https://app.cochat.ai/papers/10.1145/3640543.3645143
- The paper
- Wang, B., Li, Y., Lv, Z., Xia, H., Xu, Y., & Sodhi, R. (2024). LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing. Proceedings of the 29th International Conference on Intelligent User Interfaces. https://doi.org/10.1145/3640543.3645143
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.