CoChat
Sign Up
PreprintCited by 7Generative Adversarial Networks and Image SynthesisDigital Media and Visual ArtComputer Graphics and Visualization Techniques

The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.

PLay: Parametrically Conditioned Layout Generation using Latent Diffusion

Open access
Cheng, Chin-Yi, Huang, Forrest, Li, Gang, Li, Yang·2023·arXiv
DOI 10.48550/arxiv.2301.11529arXiv 2301.11529
Open this paper in CoChat — free accountOpen PDF

Abstract

Layout design is an important task in various design fields, including user interface, document, and graphic design. As this task requires tedious manual effort by designers, prior works have attempted to automate this process using generative models, but commonly fell short of providing intuitive user controls and achieving design objectives. In this paper, we build a conditional latent diffusion model, PLay, that generates parametrically conditioned layouts in vector graphic space from user-specified guidelines, which are commonly used by designers for representing their design intents in current practices. Our method outperforms prior works across three datasets on metrics including FID and FD-VG, and in user study. Moreover, it brings a novel and interactive experience to professional layout design processes.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2301.11529

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In design, AI has been integrated into product design workflows, fostering innovation in idea generation and advancing user interface design through adaptability.

    “In design, AI has been integrated into product design workflows, fostering innovation in idea generation [22, 39] and advancing user interface design through features such as adaptability [21, 109].”

    From Understanding Screenwriters' Practices, Attitudes, and Future Expectations in Human-AI Co-Creation (2025) · cites this paper as [21]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 51

  1. Arroyo, D. M., Postels, J., and Tombari, F. Variational trans- former networks for layout generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13642-13652, 2021.
  2. Cao, Y., Ma, Y., Zhou, M., Liu, C., Xie, H., Ge, T., and Jiang, Y. Geometry aligned variational transformer for image-conditioned layout generation. In Proceedings of the 30th ACM International Conference on Multimedia, pp. 1561-1571, 2022.
  3. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. End-to-end object detection with transformers. In European conference on computer vision, pp. 213-229. Springer, 2020.
  4. Carlier, A., Danelljan, M., Alahi, A., and Timofte, R. Deepsvg: A hierarchical generative network for vector graphics animation. Advances in Neural Information Processing Systems, 33:16351-16361, 2020.
  5. Chang, K.-H., Cheng, C.-Y., Luo, J., Murata, S., Nour- bakhsh, M., and Tsuji, Y. Building-gan: Graph- conditioned architectural volumetric design generation. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision, pp. 11956-11965, 2021.
  6. Deka, B., Huang, Z., Franzen, C., Hibschman, J., Afergan, D., Li, Y., Nichols, J., and Kumar, R. Rico: A mobile app dataset for building data-driven design applications. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, pp. 845-854, 2017.
  7. Dhariwal, P. and Nichol, A. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 34:8780-8794, 2021.
  8. Gupta, K., Lazarow, J., Achille, A., Davis, L. S., Mahade- van, V., and Shrivastava, A. Layouttransformer: Layout generation and completion with self-attention. In Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1004-1014, 2021.
  9. Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., Pritch, Y., and Cohen-Or, D. Prompt-to-prompt im- age editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022. · arXiv
  10. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
  11. Ho, J. and Salimans, T. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. · arXiv
  12. Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. Advances in Neural Information Process- ing Systems, 33:6840-6851, 2020.
  13. Ho, J., Chan, W., Saharia, C., Whang, J., Gao, R., Gritsenko, A., Kingma, D. P., Poole, B., Norouzi, M., Fleet, D. J., et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303, 2022. · arXiv
  14. Hochreiter, S. and Schmidhuber, J. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.
  15. Jayaraman, P. K., Lambourne, J. G., Desai, N., Willis, K. D., Sanghi, A., and Morris, N. J. Solidgen: An autoregres- sive model for direct b-rep synthesis. arXiv preprint arXiv:2203.13944, 2022. · arXiv
  16. Jiang, Z., Sun, S., Zhu, J., Lou, J.-G., and Zhang, D. Coarse- to-fine generative modeling for graphic layouts. In Pro- ceedings of the AAAI Conference on Artificial Intelli- gence, volume 36, pp. 1096-1103, 2022.
  17. Jyothi, A. A., Durand, T., He, J., Sigal, L., and Mori, G. Layoutvae: Stochastic scene layout generation from a label set. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9895-9904, 2019.
  18. Kingma, D. P. and Welling, M. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. · arXiv
  19. Kong, X., Jiang, L., Chang, H., Zhang, H., Hao, Y., Gong, H., and Essa, I. Blt: bidirectional layout transformer for controllable layout generation. In Computer Vision- ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XVII, pp. 474- 490. Springer, 2022.
  20. Lee, H.-Y., Jiang, L., Essa, I., Le, P. B., Gong, H., Yang, M.- H., and Yang, W. Neural design network: Graphic layout generation with constraints. In European Conference on Computer Vision, pp. 491-506. Springer, 2020.
  21. Li, G., Baechler, G., Tragut, M., and Li, Y. Learning to denoise raw mobile ui layouts for improving datasets at scale. In CHI Conference on Human Factors in Comput- ing Systems, pp. 1-13, 2022.
  22. Li, J., Yang, J., Hertzmann, A., Zhang, J., and Xu, T. Lay- outgan: Generating graphic layouts with wireframe dis- criminators. arXiv preprint arXiv:1901.06767, 2019. · arXiv
  23. Li, J., Yang, J., Zhang, J., Liu, C., Wang, C., and Xu, T. Attribute-conditioned layout gan for automatic graphic design. IEEE Transactions on Visualization and Com- puter Graphics, 27(10):4039-4048, 2020.
  24. Lin, C.-H., Gao, J., Tang, L., Takikawa, T., Zeng, X., Huang, X., Kreis, K., Fidler, S., Liu, M.-Y., and Lin, T.-Y. Magic3d: High-resolution text-to-3d content cre- ation. arXiv preprint arXiv:2211.10440, 2022. · arXiv
  25. Liu, T. F., Craft, M., Situ, J., Yumer, E., Mech, R., and Kumar, R. Learning design semantics for mobile apps. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, pp. 569-579, 2018.
  26. Louie, R., Coenen, A., Huang, C. Z., Terry, M., and Cai, C. J. Novice-ai music co-creation via ai-steering tools for deep generative models. In Proceedings of the 2020 CHI conference on human factors in computing systems, pp. 1-13, 2020.
  27. Meng, C., He, Y., Song, Y., Song, J., Wu, J., Zhu, J.-Y., and Ermon, S. Sdedit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations, 2021.
  28. Mittal, G., Engel, J., Hawthorne, C., and Simon, I. Symbolic music generation with diffusion models. arXiv preprint arXiv:2103.16091, 2021. · arXiv
  29. Monedero, J. Parametric design: a review and some ex- periences. Automation in construction, 9(4):369-377, 2000.
  30. Nauata, N., Chang, K.-H., Cheng, C.-Y., Mori, G., and Fu- rukawa, Y. House-gan: Relational generative adversarial networks for graph-constrained house layout generation. In European Conference on Computer Vision, pp. 162- 177. Springer, 2020.
  31. Nauata, N., Hosseini, S., Chang, K.-H., Chu, H., Cheng, C.-Y., and Furukawa, Y. House-gan++: Generative ad- versarial layout refinement network towards intelligent computational agent for professional architects. In Pro- ceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pp. 13632-13641, 2021.
  32. Nguyen, D. D., Nepal, S., and Kanhere, S. S. Diverse multimedia layout generation with multi choice learning. In Proceedings of the 29th ACM International Conference on Multimedia, pp. 218-226, 2021.
  33. Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., and Chen, M. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741, 2021. · arXiv
  34. Park, T., Liu, M.-Y., Wang, T.-C., and Zhu, J.-Y. Gaugan: semantic image synthesis with spatially adaptive normal- ization. In ACM SIGGRAPH 2019 Real-Time Live!, pp. 1-1. 2019.
  35. Patil, A. G., Ben-Eliezer, O., Perel, O., and Averbuch-Elor, H. Read: Recursive autoencoders for document layout generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 544-545, 2020.
  36. Perez, E., Strub, F., De Vries, H., Dumoulin, V., and Courville, A. Film: Visual reasoning with a general con- ditioning layer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
  37. Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022. · arXiv
  38. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 10684-10695, 2022.
  39. Saharia, C., Chan, W., Chang, H., Lee, C., Ho, J., Salimans, T., Fleet, D., and Norouzi, M. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 Conference Proceedings, pp. 1-10, 2022a.
  40. Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Ghasemipour, S. K. S., Ayan, B. K., Mahdavi, S. S., Lopes, R. G., et al. Photorealistic text-to-image diffusion models with deep language understanding. arXiv preprint arXiv:2205.11487, 2022b. · arXiv
  41. Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. Deep unsupervised learning using nonequi- librium thermodynamics. In International Conference on Machine Learning, pp. 2256-2265. PMLR, 2015.
  42. Strudel, R., Tallec, C., Altché, F., Du, Y., Ganin, Y., Men- sch, A., Grathwohl, W., Savinov, N., Dieleman, S., Sifre, L., et al. Self-conditioned embedding diffusion for text generation. arXiv preprint arXiv:2211.04236, 2022. · arXiv
  43. Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A. A. Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI conference on artificial intelligence, 2017.
  44. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. At- tention is all you need. Advances in neural information processing systems, 30, 2017.
  45. Xu, X., Peng, W., Cheng, C.-Y., Willis, K. D., and Ritchie, D. Inferring cad modeling sequences using zone graphs. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6062-6070, 2021.
  46. Xu, X., Willis, K. D., Lambourne, J. G., Cheng, C.-Y., Ja- yaraman, P. K., and Furukawa, Y. Skexgen: Autoregres- sive generation of cad construction sequences with dis- entangled codebooks. arXiv preprint arXiv:2207.04632, 2022. · arXiv
  47. Yamaguchi, K. Canvasvae: Learning to generate vector graphic documents. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5481- 5489, 2021.
  48. Yu, N., Chen, C.-C., Chen, Z., Meng, R., Wu, G., Josel, P., Niebles, J. C., Xiong, C., and Xu, R. Layoutdetr: De- tection transformer is a good multimodal layout designer. arXiv preprint arXiv:2212.09877, 2022. · arXiv
  49. Zhong, X., Tang, J., and Yepes, A. J. Publaynet: largest dataset ever for document layout analysis. In 2019 Inter- national Conference on Document Analysis and Recogni- tion (ICDAR), pp. 1015-1022. IEEE, 2019.
  50. Zhou, M., Xu, C., Ma, Y., Ge, T., Jiang, Y., and Xu, W. Composition-aware graphic layout gan for visual-textual presentation designs. arXiv preprint arXiv:2205.00303, 2022. · arXiv
  51. Zhu, J.-Y., Krähenbühl, P., Shechtman, E., and Efros, A. A. Generative visual manipulation on the natural image man- ifold. In European conference on computer vision, pp. 597-613. Springer, 2016.

Cited by

Sorted by citations

Metadata

Authors
Cheng, C.-Y., Huang, F., Li, G., & Li, Y.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2301.11529
arXiv
2301.11529
OpenAlex
W4318620880
Cited by
7
References
1
Citations by year
2024: 4 · 2023: 3
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Digital Media and Visual Art
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Computer Graphics and Visualization Techniques
Keywords
layout generation, latent diffusion models, vector graphics, design guidelines, user interface design, document layout, generative design, interactive design tool

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review