CoChat
Sign Up
PreprintCited by 17Generative Adversarial Networks and Image SynthesisMultimodal Machine Learning ApplicationsDigital Media and Visual Art

FIFO-Diffusion: Generating Infinite Videos from Text without Training

Open access
Kim, Jihwan, Kang, Junoh, Choi, Jinyoung, Han, Bohyung·2024·arXiv
DOI 10.48550/arxiv.2405.11473arXiv 2405.11473
Open this paper in CoChat — free accountOpen PDF

Abstract

We propose a novel inference technique based on a pretrained diffusion model for text-conditional video generation. Our approach, called FIFO-Diffusion, is conceptually capable of generating infinitely long videos without additional training. This is achieved by iteratively performing diagonal denoising, which simultaneously processes a series of consecutive frames with increasing noise levels in a queue; our method dequeues a fully denoised frame at the head while enqueuing a new random noise frame at the tail. However, diagonal denoising is a double-edged sword as the frames near the tail can take advantage of cleaner frames by forward reference but such a strategy induces the discrepancy between training and inference. Hence, we introduce latent partitioning to reduce the training-inference gap and lookahead denoising to leverage the benefit of forward referencing. Practically, FIFO-Diffusion consumes a constant amount of memory regardless of the target video length given a baseline model, while well-suited for parallel inference on multiple GPUs. We have demonstrated the promising results and effectiveness of the proposed methods on existing text-to-video generation baselines. Generated video examples and source codes are available at our project page.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2405.11473

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • FIFO-Diffusion often struggles to keep long-range temporal consistency in generated videos due to the lack of correspondence modeling across frames.

    “However, FIFO-Diffusion often struggles to keep longrange temporal consistency in the generated videos due to the lack of correspondence modeling across frames.”

    From Ouroboros-Diffusion: Exploring Consistent Content Generation in Tuning-free Long Video Diffusion (2025) · cites this paper as Kim et al. 2024

    Section 3.1 (p. 4) says FIFO-Diffusion propagates context to later frames and lets each frame reference preceding frames, extending consistency to longer sequences. The qualitative results (p. 6) report that it maintains motion consistency by referencing earlier frames. The paper attributes difficulty maintaining long-term context to chunked autoregressive methods, not FIFO-Diffusion; it does not identify a lack of cross-frame correspondence modeling as the cause. (LLM, conf=0.88)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • FIFO-Diffusion fails to utilize visual information contained in earlier denoised frames, which exacerbates temporal inconsistency in continuous video diffusion.

    “Moreover, FIFO-Diffusion fails to utilize visual information contained in the earlier denoised frames, which exacerbates temporal inconsistency in continuous video diffusion.”

    From Ouroboros-Diffusion: Exploring Consistent Content Generation in Tuning-free Long Video Diffusion (2025) · cites this paper as Kim et al. 2024

    Section 3.1 (p. 4) says the opposite: FIFO-Diffusion propagates context to later frames, lets each frame reference preceding frames, and thereby extends local consistency to longer sequences. Section 1 (p. 2) also says its lookahead denoising lets frames reference cleaner frames. The paper attributes inadequate contextual information and resulting discontinuities to chunked autoregressive methods, not FIFO-Diffusion. (LLM, conf=0.89)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • The latent enqueue-dequeue cycle allows for the incremental generation of video frames.

    “The latent enqueue-dequeue cycle allows for the incremental generation of video frames.”

    From Ouroboros-Diffusion: Exploring Consistent Content Generation in Tuning-free Long Video Diffusion (2025) · cites this paper as Kim et al. 2024

  • Once frames are fully denoised, they are dequeued and no longer contribute to the generation of subsequent frames.

    “Once frames are fully denoised, they are dequeued and no longer contribute to the generation of subsequent frames.”

    From Ouroboros-Diffusion: Exploring Consistent Content Generation in Tuning-free Long Video Diffusion (2025) · cites this paper as Kim et al. 2024

  • The tail latent is typically sampled from a standard Gaussian distribution, which contains pure noise without any visual information.

    “The tail latent is typically sampled from a standard Gaussian distribution, which contains pure noise without any visual information.”

    From Ouroboros-Diffusion: Exploring Consistent Content Generation in Tuning-free Long Video Diffusion (2025) · cites this paper as Kim et al. 2024

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 37

  1. [1][1] Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In CVPR , 2023. -
  2. [2][2] Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, Chao Weng, and Ying Shan. VideoCrafter1: Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310.19512 , 2023. - · arXiv
  3. [3][3] Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. VideoCrafter2: Overcoming data limitations for high-quality video diffusion models. arXiv preprint arXiv:2401.09047 , 2024. - · arXiv
  4. [4][4] Xinyuan Chen, Yaohui Wang, Lingjun Zhang, Shaobin Zhuang, Xin Ma, Jiashuo Yu, Yali Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. Seine: Short-to-long video diffusion model for generative transition and prediction. arXiv preprint arXiv:2310.20700 , 2023. - · arXiv
  5. [5][5] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis. In NeurIPS , 2021. -
  6. [6][6] William Harvey, Saeid Naderiparizi, Vaden Masrani, Christian Weilbach, and Frank Wood. Flexible diffusion modeling of long videos. In NeurIPS , 2022. -
  7. [7][7] Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity long video generation. arXiv preprint arXiv:2211.13221 , 2022. - · arXiv
  8. [8][8] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS , 2020. -
  9. [9][9] Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video diffusion models. In NeurIPS , 2022. -
  10. [10][10] Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. CogVideo: Large-scale pretraining for text-to-video generation via transformers. In ICLR , 2023. -
  11. [11][11] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In NeurIPS , 2022. -
  12. [12][12] Zhengxiong Luo, Dayou Chen, Yingya Zhang, Yan Huang, Liang Wang, Yujun Shen, Deli Zhao, Jingren Zhou, and Tieniu Tan. Videofusion: Decomposed diffusion models for high-quality video generation. In CVPR , 2023. -
  13. [13][13] Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. arXiv preprint arXiv:2401.03048 , 2024. - · arXiv
  14. [14][14] Kangfu Mei and Vishal M. Patel. Vidm: Video implicit diffusion models. In AAAI , 2023. -
  15. [15][15] OpenAI. GPT-4 technical report. arXiv preprint arXiv:2303.08774 , 2023. - · arXiv
  16. [16][16] William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV , 2023. -
  17. [17][17] Haonan Qiu, Menghan Xia, Yong Zhang, Yingqing He, Xintao Wang, Ying Shan, and Ziwei Liu. FreeNoise: Tuning-free longer video diffusion via noise rescheduling. arXiv preprint arXiv:2310.15169 , 2023. - · arXiv
  18. [18][18] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR , 2022. -
  19. [19][19] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional networks for biomedical image segmentation. In MICCAI , 2015. -
  20. [20][20] David Ruhe, Jonathan Heek, Tim Salimans, and Emiel Hoogeboom. Rolling diffusion models. In ICML , 2024. -
  21. [21][21] Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In NeurIPS , 2016. -
  22. [22][22] Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. Make-A-Video: Text-to-video generation without text-video data. In ICLR , 2022. -
  23. [23][23] Ivan Skorokhodov, Sergey Tulyakov, and Mohamed Elhoseiny. Stylegan-v: A continuous video generator with the price, image quality and perks of stylegan2. In CVPR , 2022. -
  24. [24][24] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR , 2021. -
  25. [25][25] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR , 2021. -
  26. [26][26] Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402 , 2012. - · arXiv
  27. [27][27] Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.0171 , 2018. -
  28. [28][28] Ruben Villegas, Mohammad Babaeizadeh, Pieter-Jan Kindermans, Hernan Moraldo, Han Zhang, Mohammad Taghi Saffar, Santiago Castro, Julius Kunze, and Dumitru Erhan. Phenaki: Variable length video generation from open domain textual description. In ICLR , 2023. -
  29. [29][29] Vikram Voleti, Alexia Jolicoeur-Martineau, and Christopher Pal. MCVD: Masked conditional video diffusion for prediction, generation, and interpolation. In NeurIPS , 2022. -
  30. [30][30] Fu-Yun Wang, Wenshuo Chen, Guanglu Song, Han-Jia Ye, Yu Liu, and Hongsheng Li. GenL-Video: Multi-text to long video generation via temporal co-denoising. arXiv preprint arXiv:2305.18264 , 2023. - · arXiv
  31. [31][31] Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. ModelScope text-to-video technical report. arXiv preprint arXiv:2308.06571 , 2023. - · arXiv
  32. [32][32] Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, Yuwei Guo, Tianxing Wu, Chenyang Si, Yuming Jiang, Cunjian Chen, Chen Change Loy, Bo Dai, Dahua Lin, Yu Qiao, and Ziwei Liu. LaVie: High-quality video generation with cascaded latent diffusion models. arXiv preprint arXiv:2309.15103 , 2023. - · arXiv
  33. [33][33] Jun Xu, Tao Mei, Ting Yao, and Yong Rui. MSR-VTT: A large video description dataset for bridging video and language. In CVPR , 2016. -
  34. [34][34] Shengming Yin, Chenfei Wu, Huan Yang, Jianfeng Wang, Xiaodong Wang, Minheng Ni, Zhengyuan Yang, Linjie Li, Shuguang Liu, Fan Yang, Jianlong Fu, Gong Ming, Lijuan Wang, Zicheng Liu, Houqiang Li, and Nan Duan. NUWA-XL: Diffusion over diffusion for extremely long video generation. arXiv preprint arXiv:2303.12346 , 2023. - · arXiv
  35. [35][35] Sihyun Yu, Kihyuk Sohn, Subin Kim, and Jinwoo Shin. Video probabilistic diffusion models in projected latent space. In CVPR , 2023. -
  36. [36][36] Zihan Zhang, Richard Liu, Kfir Aberman, and Rana Hanocka. Tedi: Temporally-entangled diffusion for long-term motion synthesis. In SIGGRAPH , 2024. -
  37. [37][37] Daquan Zhou, Weimin Wang, Hanshu Yan, Weiwei Lv, Yizhe Zhu, and Jiashi Feng. MagicVideo: Efficient video generation with latent diffusion models. arXiv preprint arXiv:2211.11018 , 2022. · arXiv

Cited by

Sorted by citations

Authors
Kim, J., Kang, J., Choi, J., & Han, B.
Venue
arXiv
Publisher
arXiv
Published
2024
Pages
89834-89868
ISSN
1049-5258 (ISSN-L)
DOI
10.48550/arxiv.2405.11473
arXiv
2405.11473
OpenAlex
W4415796513
Cited by
17
References
0
Citations by year
2026: 13 · 2025: 4
FWCI
3.52
Type
conference paper
Language
en
Open access
Licence
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Digital Media and Visual Art
Keywords
text-to-video generation, diffusion models, training-free generation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review