Manga Generation via Layout-controllable Diffusion
Open accessAbstract
Generating comics through text is widely studied. However, there are few studies on generating multi-panel Manga (Japanese comics) solely based on plain text. Japanese manga contains multiple panels on a single page, with characteristics such as coherence in storytelling, reasonable and diverse page layouts, consistency in characters, and semantic correspondence between panel drawings and panel scripts. Therefore, generating manga poses a significant challenge. This paper presents the manga generation task and constructs the Manga109Story dataset for studying manga generation solely from plain text. Additionally, we propose MangaDiffusion to facilitate the intra-panel and inter-panel information interaction during the manga generation process. The results show that our method particularly ensures the number of panels, reasonable and diverse page layouts. Based on our approach, there is potential to converting a large amount of textual stories into more engaging manga readings, leading to significant application prospects.
AI snapshot
Written from the paper’s full text.
TL;DR
This paper introduces the task of generating a multi-panel manga page directly from a plain-text story, along with the Manga109Story dataset and the MangaDiffusion model. MangaDiffusion generates all panels in one pass, controls panel count and layout, and uses masking to suppress unwanted speech bubbles.
Key findings
- MangaDiffusion reaches FID 143.5 and CLIP-I 83.6 on the Manga109Story test set, compared with fine-tuned Pixart-Σ at FID 149.1 and CLIP-I 84.6; without the speech-bubble masking strategy, MangaDiffusion scores FID 179.8 and CLIP-I 75.7.
- The method maintains the requested number of panels, produces varied and non-uniform page layouts, and shows character consistency across panels through inter-panel information exchange.
- The constructed Manga109Story dataset provides per-panel captions and a page-level story summary, built from Manga109’s 109 volumes and 21,142 pages; the experiments use one title with 163 pages for testing and 108 titles with 20,979 pages for training.
- Qualitative comparisons indicate that general text-to-image models often have better image quality but weaker control over panel count and story correspondence.
Methods
- The pipeline first uses an LLM to split a user’s plain-text story into K script segments, then feeds those scripts into MangaDiffusion to generate K ordered panels; the final page is formed by taking the pixel-wise minimum across the generated panels.
- MangaDiffusion encodes each panel image with a pretrained VAE, patchifies the latents, and processes them through transformer blocks that alternate between within-panel attention and across-panel attention; each panel’s caption controls only that panel, and padding panels are masked out.
- Training uses a maximum of 8 panels, discards pages with more than 8 panels, pads shorter pages with white panels labeled “EMPTY,” and applies an intra-panel mask from a speech-bubble segmentation model so text and bubble regions are excluded from attention and loss. The model is initialized from Latte, uses Stable Diffusion 1.4 VAE and T5 text encoder, trains for 100,000 steps on 4 NVIDIA A100 GPUs for 65 hours, with batch size 8, learning rate 1e-4, and images resized to 512×384.
Limitations
- The paper states that manga data collection is difficult because of intellectual property restrictions, and that Manga109, while the largest open-source manga dataset known to the authors, has only 21,142 pages, far fewer than large-scale image datasets.
- The authors state that MangaDiffusion still has substantial room to improve character consistency and consistency between panels and captions.
- The text makes evident additional constraints: evaluation uses only FID and CLIP-I, with CLIP text-image similarity omitted because captions exceed CLIP’s token limit; the test set is a single manga title; and the model is limited to at most 8 panels per page.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
7 of 9 cited sources checked · 1 supported · 0 flagged · accuracy not yet measured
- Findingcitation
The Manga109Story dataset is constructed based on the existing Manga109 dataset.
“We construct the Manga109Story dataset by using the existing Manga109 dataset [34]”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [34] 34. Xichen Pan, Pengda Qin, Yuhong Li, Hui Xue, and Wenhu Chen. Synthesizing coherent story with auto-regressive la- tent diffusion models. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision, pages 2920-2930, 2024. 2 source
Section 4.1 (page unavailable) identifies PororoSV, FlintstonesSV, and VIST as the paper’s datasets. Searches for “Manga109Story” and “Manga109” found no passage addressing Manga109 or the construction of Manga109Story, so our copy cannot verify the cited dataset identification. (LLM, conf=0.76)
Experimental — accuracy not yet measured
- Findingown data
Removing speech bubbles affects the performance of MangaDiffusion.
“removing speech bubbles affects the performance”
Quote in the paperSubstantial claim - Findingown data
The Manga109Story dataset provides detailed captioning for each manga page using a multimodal large language model (MLLM).
“we introduce the Manga109Story dataset, which builds upon previous excellent work and employs the multi-modal understanding capability of MLLM to provide detailed captioning for each manga page.”
Quote in the paperSubstantial claim- Rests on an assumption: The MLLM can accurately understand manga panels and generate detailed captions.
- Findingown data
Training with speech bubble masks reduces the number of speech bubbles in generated manga.
“By adding the intra-panel mask during training, the generated results of the model greatly reduced the number of speech bubbles”
Quote in the paperSubstantial claim - Findingown data
Non-fine-tuned T2I models have poor FID and CLIP-I performance on manga data.
“For the T2I models that have not been finetuned, theirbFID and CLIP-I performances are poor.”
Quote in the paperSubstantial claim
- Findingnone given
The MangaDiffusion network utilizes transformer blocks for intra-panel and inter-panel information interaction.
“we propose the MangaDiffusion network, which utilizes transformer blocks for intra-panel and inter-panel information interaction”
Quote in the paperSubstantial claim - Findingreasoning
MangaDiffusion can maintain character consistency between different panels on the same page.
“our method can maintain character consistency between different panels on the same page”
Quote in the paperSubstantial claim - Findingown data
MangaDiffusion performs well in terms of FID and CLIP-I on the Manga109Story test set.
“From the table, it can be seen that our model performs well in terms of FID and CLIP-I.”
Quote in the paperSubstantial claim - Findingnone given
The proposed intra-panel block enables independent control of the generation of different panels within the same image.
“Our proposed intra-panel block enables independent control of the generation of different panels within the same image.”
Quote in the paperNot a substantial claim - Findingnone given
MangaDiffusion ensures the number of panels generated in the manga page is consistent with the corresponding prompt input.
“MangaDiffusion ensures that the number of panels generated in the manga page is consistent with the corresponding prompt input”
Quote in the paperSubstantial claim - Findingnone given
The MangaDiffusion model facilitates intra-panel and inter-panel information interaction during manga generation.
“we propose MangaDiffusion to facilitate the intra-panel and inter-panel information interaction during the manga generation process.”
Quote in the paperSubstantial claim - Findingown data
Each panel generated by MangaDiffusion depicts a story that has good relevance to the prompt.
“each panel depicts a story that has good relevance to the prompt.”
Quote in the paperSubstantial claim - Findingown data
The arrangement of the content of each panel is distributed in reading order.
“We can see that the arrangement of the content of each panel is distributed in reading order”
Quote in the paperSubstantial claim - Findingown data
In overlapping panels, the overlapped content between panels remains almost identical.
“the overlapped content between panels remains almost identical”
- Also stated in: § Additional information · plain
“the contents of the overlapping regions between panels are almost identical”
Quote in the paperSubstantial claim - Also stated in: § Additional information · plain
- Interpretationown data
The inter-panel block effectively facilitates information interaction between panels.
“This suggests that the inter-panel block effectively facilitates the information interaction between panels”
Quote in the paperSubstantial claim - Interpretationown data
The model has learned a reasonable layout.
“indicating that our model has learned a reasonable layout.”
Quote in the paperSubstantial claim - Interpretationown data
The method enables effective information interaction between panels.
“This also demonstrates that our method enables effective information interaction between panels.”
Quote in the paperNot a substantial claim - Premisenone given
There is currently no research on directly generating manga from plain text.
“To our knowledge, there is currently no research on directly generating manga from plain text.”
Quote in the paperNot a substantial claim - Interpretationreasoning
The proposed approach mitigates the problem of incomplete objectives in multi-object generation tasks.
“thus mitigating the problem of incomplete objectives in multi-object generation tasks.”
Quote in the paperSubstantial claim- Rests on an assumption: Current generation models often face the issue of incomplete objectives in multi-object generation.
- Interpretationown data
The inter-panel block presents precise complementarity in the visual contents of each panel.
“thereby presenting the precise complementarity in the visual contents of each panel.”
Quote in the paperSubstantial claim
- Classified as probably not a substantial claim. 21
- May combine two claims. 12
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe Manga109 dataset includes coordinates of panels, characters, texts, and contents of dialogues.
“including the coordinates of the bounding boxes for each frame (referred to as panel in this paper), the names and coordinates of each character, the coordinates of each face, and the coordinates and content of each text.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [34] 34. Xichen Pan, Pengda Qin, Yuhong Li, Hui Xue, and Wenhu Chen. Synthesizing coherent story with auto-regressive la- tent diffusion models. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision, pages 2920-2930, 2024. 2 source
Section 4.1 (Datasets; page not provided) describes PororoSV, FlintstonesSV, and VIST, but does not address Manga109 or its annotations. Searches for “Manga109” and for panel, character, face, and text coordinates found no passage supporting the attributed description. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationManga109 is widely used in the fields of image super-resolution and image restoration.
“This dataset is widely used in the fields of image super-resolution and image restoration.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [34] 34. Xichen Pan, Pengda Qin, Yuhong Li, Hui Xue, and Wenhu Chen. Synthesizing coherent story with auto-regressive la- tent diffusion models. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision, pages 2920-2930, 2024. 2 source
The cited paper’s §4.1 (Datasets; page unavailable) identifies PororoSV, FlintstonesSV, and VIST as its testbeds, but does not address Manga109. Searches for Manga109, super-resolution, and image restoration found no passage supporting the claimed widespread use. (LLM, conf=0.75)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationManga109 consists of 109 volumes of manga drawn by professional manga artists in Japan, totaling 21,142 pages.
“Manga109 consists of 109 volumes of manga drawn by professional manga artists in Japan, totaling 21,142 pages.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [34] 34. Xichen Pan, Pengda Qin, Yuhong Li, Hui Xue, and Wenhu Chen. Synthesizing coherent story with auto-regressive la- tent diffusion models. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision, pages 2920-2930, 2024. 2 source
Section 4.1 (Datasets; page unavailable) describes PororoSV, FlintstonesSV, and VIST, not Manga109. Searches for Manga109 and “21,142 pages” found no passage in our copy supporting the claimed number of volumes, artists, or pages. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationTimestep t is injected into the model using adaLN-single.
“Timestep t is injected into the model using adaLN-single[7].”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [7] 7. Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-α: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310.00426, 2023. 5 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationManga109 is currently the largest open-source manga dataset.
“Manga109 is currently the largest opensource manga dataset and it offers rich annotation resources that can be utilized”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [34] 34. Xichen Pan, Pengda Qin, Yuhong Li, Hui Xue, and Wenhu Chen. Synthesizing coherent story with auto-regressive la- tent diffusion models. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision, pages 2920-2930, 2024. 2 source
The Datasets section (4.1; page not provided) identifies PororoSV, FlintstonesSV, and VIST as the paper’s testbeds. Searches for Manga109 and manga annotations found no passage addressing whether Manga109 is open-source, the largest manga dataset, or richly annotated. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Classified as probably not a substantial claim. May combine two claims.
FindingcitationMLLM (GPT-4o) is used to generate captions for each panel and summarize a story for each page.
“we utilize the powerful multimodal understanding ability of MLLM (GPT-4o [1]) to fed the original manga page and the corresponding XML file into MLLM, and prompt it to generate corresponding captions for each panel and summarize the corresponding story.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [1] 1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. 4 source
The Introduction (p. 1) describes GPT-4 as a multimodal model that accepts images and text and produces text; the visual-input example (p. 9) shows it describing an image panel by panel. The cited report identifies GPT-4, not the specifically cited GPT-4o. The manga-captioning and story-summary workflow is the citing authors’ own method, not a finding attributed to this report. (LLM, conf=0.85)
Experimental — accuracy not yet measured
May combine two claims.
Findingown dataThe T2I models have better image quality than MangaDiffusion, but their performance in panel quantity and storyline is not outstanding.
“Although the T2I models have better image quality, their performance in terms of panel quantity and storyline is not outstanding.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenThe inter-panel mask masks out padding images.
“the inter-panel mask is used to mask out the padding images.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe Manga109Story dataset provides captions for each panel and a story for the entire page.
“In the final Manga109Story dataset, we provide captions for each panel and a story for the entire page.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Findingnone givenThe intra-panel mask strategy reduces speech bubble generation and improves the quality of generated manga.
“Proposing a local mask strategy to reduce speech bubble generation and improve the quality of generated manga.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataMangaDiffusion produces reasonable and diverse page layouts.
“The results show that our method particularly ensures the number of panels, reasonable and diverse page layouts.”
- Also stated in: § 5.3. Qualitative Results · plain
“MangaDiffusion's panel layout is flexible and diverse, not a rigid and uniform layout that is reflected in the results of other methods.”
Quote in the paperSubstantial claim- Also stated in: § 5.3. Qualitative Results · plain
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenThe caption only participates in the computation within the intra-panel block.
“The caption only participates in the computation within the intra-panel block.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingnone givenIn the intra-panel mask, tokens in the white region are involved in attention and loss calculation, while black regions are not involved.
“The tokens in the white region are involved in attention calculation and loss calculation, while the black regions are not involved.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Findingnone givenThe MangaDiffusion model allows interaction and perception of information among panels during the generation process.
“Introducing the MangaDiffusion method, which allows interaction and perception of information among panels during the generation process.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenThe intra-panel mask removes text and speech bubble boxes within the image.
“The intra-panel mask is used to remove text and speech bubble boxes within the image”
Quote in the paperNot a substantial claim- Rests on an assumption: The intra-panel mask accurately identifies text and speech bubble regions.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenMangaDiffusion currently has great room for development in character consistency and consistency between panels and captions.
“The MangaDiffusion we propose currently has great room for development in these two aspects of consistency.”
Quote in the paperNot a substantial claimMay combine two claims.
FindingreasoningThe network learns that padding panels do not contain any visuals and do not participate in the layout.
“Through these efforts, the network learns that the padding panels do not contain any visuals and do not participate in the layout.”
Quote in the paperSubstantial claimMay combine two claims.
Findingnone givenMangaDiffusion learns the layout information between panels, enabling generation of K panels in a reasonable order while ensuring flexibility in panel layout.
“After training, our model is able to learn the layout information between panels, enabling the generation of K panels in a reasonable order while ensuring flexibility in panel layout.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
FindingreasoningThe images generated by non-fine-tuned T2I models have high image quality.
“However, the images generated by these models have high image quality”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe method performs well in terms of panel quantity control and panel layout diversity.
“It can be observed from the figures that our method performs well in terms of panel quantity control and panel layout diversity.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Findingnone givenBy inputting the original manga page and XML file into the MLLM, and following the prompts, captions for each panel and a summarized story are generated.
“By inputting the original manga page and XML file into the MLLM, and following the prompts, captions for each panel and a summarized story are generated.”
Quote in the paperSubstantial claim- Rests on an assumption: The MLLM can accurately understand manga panels and generate detailed captions.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenThe transformer block consists of an intra-panel block and an inter-panel block for information interaction.
“The transformer block consists of an intra-panel block and an inter-panel block for information interaction.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Findingnone givenThe XML file for Manga109Story is synthesized by integrating data from the Manga109 dataset, the Manga109Dialog dataset, and the results from the panel order estimator.
“The presented XML file is synthesized by integrating the data from the Manga109 dataset, the Manga109Dialog dataset, and the results from the panel order estimator.”
Quote in the paperSubstantial claim- Rests on an assumption: The panel order estimator accurately estimates the reading order of panels.
May combine two claims.
Findingown dataThe MangaDiffusion network allows flexible and diverse layouts in generated manga pages while maintaining control over the number of panels.
“allowing for flexi-ble and diverse layouts in the generated manga pages while maintaining control over the number of panels.”
- Also stated in: § XML File Generated Captions · boosted
“It can be observed from the figures that our method performs well in terms of panel quantity control and panel layout diversity.”
Quote in the paperSubstantial claim- Also stated in: § XML File Generated Captions · boosted
May combine two claims.
Premisenone givenThe total number of pages in Manga109 is only 21,142, far less than other large-scale image datasets.
“However, the total number of pages is only 21,142, which is far less than other large-scale image datasets.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenThe Manga109 dataset used in this paper is the largest open-source dataset known to us.
“The Manga109 dataset used in this paper is the largest open-source dataset known to us.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationManga109Dialog is a large-scale dialogue dataset for comics speaker detection.
“Manga109Dialog [25] is a large-scale dialogue dataset for comics speaker detection.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [25] 25. Zhansheng Li, Nanxuan Zhao, Zongwei Wu, Yihua Dai, Junle Wang, Yanqing Jing, and Shengfeng He. Reference- based screentone transfer via pattern correspondence and regularization. In Computer Graphics Forum, page e14800. Wiley Online Library, 2023. 3
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
Premisenone givenCollecting manga data is particularly difficult because manga is protected by intellectual property rights.
“collecting manga data is particularly difficult because manga is protected by intellectual property rights, which imposes corresponding restrictions on the collection and use of manga data.”
Quote in the paperSubstantial claim- Rests on an assumption: Manga is protected by intellectual property rights, which imposes restrictions on the collection and use of manga data.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe Manga109 community offers a panel order estimator that predicts the order of manga panels.
“Manga109 community offers a panel order estimator which uses the method proposed in [20] to predict the order of manga panels.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [20] 20. Samu Kovanen and Kiyoharu Aizawa. A layered method for determining manga text bubble reading order. In 2015 IEEE International Conference on Image Processing (ICIP), pages 4283-4287, 2015. 2, 3 source
The paper relies on this without support.
Japanese manga contains multiple panels on a single page, with characteristics such as coherence in storytelling, reasonable and diverse page layouts, consistency in characters, and semantic correspondence between panel drawings and panel scripts.
“Japanese manga contains multiple panels on a single page, with characteristics such as coherence in storytelling, reasonable and diverse page layouts, consistency in characters, and semantic correspondence between panel drawings and panel scripts.”
The paper relies on this without support.
Manga generation has considerable potential in design, business, and other fields.
“Manga generation has considerable potential in design, business, and other fields.”
The paper relies on this without support.
It has the potential to convert a massive amount of plain textual content into more engaging manga materials, thus bringing significant commercial prospects.
“It has the potential to converting a massive amount of plain textual content into more engaging manga materials, thus bringing significant commercial prospects.”
The paper relies on this without support.
Current generation models often face the issue of incomplete objectives in multi-object generation.
“current generation models often face the issue of incomplete objectives.”
The paper relies on this without support.
More training data is required to train a manga generation model with better performance under the scaling law.
“Under the guidance of the scaling law, in order to train a manga generation model with better performance, more training data is undoubtedly required.”
The paper relies on this without support.
Manga is protected by intellectual property rights, which imposes restrictions on the collection and use of manga data.
“manga is protected by intellectual property rights, which imposes corresponding restrictions on the collection and use of manga data.”
The paper relies on this without support.
The MLLM can accurately understand manga panels and generate detailed captions.
“employs the multi-modal understanding capability of MLLM to provide detailed captioning for each manga page”
“By inputting the original manga page and XML file into the MLLM, and following the prompts, captions for each panel and a summarized story are generated.”
The paper relies on this without support.
The panel order estimator accurately estimates the reading order of panels.
“the results from the panel order estimator”
The paper relies on this without support.
The intra-panel mask accurately identifies text and speech bubble regions.
“The intra-panel mask is used to remove text and speech bubble boxes within the image”
“The tokens in the white region are involved in attention calculation and loss calculation, while the black regions are not involved.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Manga109Dialog citation points to screentone transfer
MinorThe cited entry 25 is about screentone transfer, not the dialogue dataset, which appears as entry 24.
“Manga109Dialog [25] is a large-scale dialogue dataset for comics speaker detection.”
“25. Zhansheng Li, Nanxuan Zhao, Zongwei Wu, Yihua Dai, Junle Wang, Yanqing Jing, and Shengfeng He. Reference- based screentone transfer via pattern correspondence and regularization.”
FLUX table citation points to story visualization
MinorThe table labels FLUX with reference 21, whose entry is a word-level story-visualization paper rather than FLUX.
“FLUX[21] 229.6 71.0”
“21. Bowen Li. Word-level fine-grained story visualization.”
Example changes a character's name between input and script
MinorThe example user input names the character Shunpei, while its generated panel script calls him Junpei.
“Shunpei and Miki find themselves tumbling down a hill in the forest, overwhelmed by the sudden event.”
“Onlookers Junpei and Miki are seen running through a dense forest.”
Caption conditioning is assigned to two incompatible locations
MinorThe architecture description says text embeddings condition both types of transformer block, then says captions condition only the intra-panel calculation.
“the text embedding z T is used as the control condition for the intra-panel block and inter-panel block calculation as shown in the Figure 4. Captions are used as control conditions only within the intra-panel calculation.”
“The caption only participates in the computation within the intra-panel block.”
Latte backbone citation points to a different model
MinorThe implementation cites reference 29 for Latte, but entry 29 is a story-visualization paper; Latte appears as entry 28.
“Our model is modified based on the Latte [29]”
“29. Adyasha Maharana and Mohit Bansal. Integrating visuospa- tial, linguistic and commonsense structure into story visual- ization.”
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 16
- [4]4. Ying Cao, Antoni B Chan, and Rynson WH Lau. Auto- matic stylistic manga layout. ACM Transactions on Graphics (TOG), 31(6):1-10, 2012.
- [12]12. Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learn- ing, 2024. 7
- [15]15. Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020. 2, 6
- [21]21. Bowen Li. Word-level fine-grained story visualization. In European Conference on Computer Vision, pages 347-362. Springer, 2022. 2
- [22]22. Hua Li and David Mould. Content-sensitive screening in black and white. In International Conference on Com- puter Graphics Theory and Applications, pages 166-172. SCITEPRESS, 2011. 3
- [23]23. Yitong Li, Zhe Gan, Yelong Shen, Jingjing Liu, Yu Cheng, Yuexin Wu, Lawrence Carin, David Carlson, and Jianfeng Gao. Storygan: A sequential conditional gan for story vi- sualization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6329-6338, 2019. 2
- [24]24. Yingxuan Li, Kiyoharu Aizawa, and Yusuke Matsui. Manga109dialog: A large-scale dialogue dataset for comics speaker detection. In Proceedings of the IEEE International Conference on Multimedia and Expo, 2024. 2, 3
- [25]25. Zhansheng Li, Nanxuan Zhao, Zongwei Wu, Yihua Dai, Junle Wang, Yanqing Jing, and Shengfeng He. Reference- based screentone transfer via pattern correspondence and regularization. In Computer Graphics Forum, page e14800. Wiley Online Library, 2023. 3
- [37]37. Yingge Qu, Tien-Tsin Wong, and Pheng-Ann Heng. Manga colorization. ACM Transactions on Graphics (ToG), 25(3): 1214-1220, 2006. 3
- [38]38. Yingge Qu, Wai-Man Pang, Tien-Tsin Wong, and Pheng- Ann Heng. Richness-preserving manga screening. ACM Transactions on Graphics (TOG), 27(5):1-8, 2008. 3
- [46]46. Daniel Sỳkora, John Dingliana, and Steven Collins. Lazy- brush: Flexible painting tool for hand-drawn cartoons. In Computer Graphics Forum, pages 599-608. Wiley Online Library, 2009. 3
- [47]47. Takamasa Tanaka, Kenji Shoji, Fubito Toyama, and Juichi Miyamichi. Layout analysis of tree-structured scene frames in comic images. In IJCAI, pages 2885-2890. Citeseer, 2007.
- [49]49. Huisi Wu, Ziheng Ma, Wenliang Wu, Xueting Liu, Chengze Li, and Zhenkun Wen. Shading-guided manga screening from reference. IEEE Transactions on Visualization and Computer Graphics, 2023. 3
- [50]50. Zhipeng Wu and Kiyoharu Aizawa. Mangawall: Generat- ing manga pages for real-time applications. In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 679-683. IEEE, 2014. 3
- [52]52. Tao Xu, Pengchuan Zhang, Qiuyuan Huang, Han Zhang, Zhe Gan, Xiaolei Huang, and Xiaodong He. Attngan: Fine- grained text to image generation with attentional generative adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1316- 1324, 2018. 2
- [54]54. Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiao- gang Wang, Xiaolei Huang, and Dimitris N Metaxas. Stack- gan++: Realistic image synthesis with stacked generative ad- versarial networks. IEEE transactions on pattern analysis and machine intelligence, 41(8):1947-1962, 2018. 2
Cited by
Sorted by citations
Generative AI for Cel-Animation: A Survey
Tang, Y., Guo, J., Liu, P., Wang, Z., Hua, H., Zhong, J.-X., Xiao, Y., Huang, C., Song, L., Liang, S., et al. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00400 · Cited by 19 · see citing works
“…[62], StoryGen [66], CogCartoon [157], SGSIG [148], VideoComposer [122], LayoutGAN [60], Layout2Vid [64], DiffSensei [129], MangaDiffusion [16], CameraCtrl [36] (§3.2) AnimateAnyone [39], Champ [156], MimicMotion [149], Animate-X [108], MikuDance [144], CoNR [65] Inbetweening (§3.2)…”
InfluentialViStoryBench: Comprehensive Benchmark Suite for Story Visualization
Zhuang, C., Huang, A., Hu, Y., Wu, J., Cheng, W., Liao, J., Wang, H., Liao, X., Cai, W., Xu, H., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.24862
“In story video generation [23, 31, 32, 37, 81, 102, 104, 107, 108], systems like MMStoryAgent [96], Ani-mDirector [43], MovieAgent [85] and FilmMaster [33] orchestrate multi-component pipelines.”
MangaFlow: An End-to-End Agentic Framework for Controllable Story to Manga Generation
Wang, M., Xie, Z., Chen, Y., Xiu, L., & Nakayama, H. (2026) · arXiv · doi:10.48550/arxiv.2605.28173
“More recently, MangaDiffusion (Chen et al., 2024) formulates text-only manga generation and constructs Manga109Story to study multi-panel manga page generation from plain text, while DiffSen-sei (Wu et al., 2025) introduces customized manga generation with multi-character control and layout-aware…”
InfluentialKwon, P., & Chen, C. (2025) · arXiv · doi:10.48550/arxiv.2512.01686
“Comic generation [8, 62] is also a notable subtask within story visualization.”
LayoutShop: Content-Constrained Exploratory Design of Creative Article Layout
Li, J.-L., & Xu, P. (2026) · doi:10.48550/arxiv.2609.05098
OnomatoGen: Onomatopoeia Generation with the Alpha-Channel in Manga
Taniguchi, T., Shimoda, W., Yamaguchi, K., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2509.24331
“Manga generation methods [4, 18] and manga-related computer vision meth-ods [11–13] should be able to alleviate overwork and support faster content supply.”
Visually Grounded Narratives: Reducing Cognitive Burden in Researcher-Participant Interaction
Wu, R.-T., Song, J.-Y., Teng, F., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2509.00381
“Such capabilities have led some studies to extend its application into the field of story generation [46] [47] [48] [49], demonstrating its potential in generating multimodal narrative content.”
- Authors
- Chen, S., Li, D., Bao, Z., Zhou, Y., Tan, L., Zhong, Y., & Zhao, Z.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2412.19303
- OpenAlex
- W4405902429
- Semantic Scholar
- 13840f3a30b6cd4c3ebdb7cc1603363b3948b0de
- Cited by
- 0
- References
- 0
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Social Sciences › Arts and Humanities › Visual Arts and Performing Arts › Comics and Graphic Narratives
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Digital Media and Visual Art
- Keywords
- text-to-image generation, diffusion models, manga
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review