Exploiting the Signal-Leak Bias in Diffusion Models
Open accessAbstract
There is a bias in the inference pipeline of most diffusion models. This bias arises from a signal leak whose distribution deviates from the noise distribution, creating a discrepancy between training and inference processes. We demonstrate that this signal-leak bias is particularly significant when models are tuned to a specific style, causing sub-optimal style matching. Recent research tries to avoid the signal leakage during training. We instead show how we can exploit this signal-leak bias in existing diffusion models to allow more control over the generated images. This enables us to generate images with more varied brightness, and images that better match a desired style or color. By modeling the distribution of the signal leak in the spatial frequency and pixel domains, and including a signal leak in the initial latent, we generate images that better match expected results without any additional training.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The low-frequency component preserves layout information for overall video consistency.
“The low-frequency component preserves layout information for overall video consistency, while the high-frequency component introduces necessary video dynamics.”
From Ouroboros-Diffusion: Exploring Consistent Content Generation in Tuning-free Long Video Diffusion (2025) · cites this paper as Everaert et al. 2024
Section 3.2 says low-frequency information from an image can survive diffusion noise, and Section 5.4 shows that selecting low-frequency components can influence generated images. The paper discusses image brightness and color, however, not preservation of layout across video frames or overall video consistency. (LLM, conf=0.82)
Experimental — accuracy not yet measuredThe low-frequency component in latent space primarily corresponds to layout and overall structure in pixel space.
“Recent advances (Wu et al. 2023b;Everaert et al. 2024) indicate that the lowfrequency component in the latent space primarily corresponds to the layout and overall structure in the pixel space.”
From Ouroboros-Diffusion: Exploring Consistent Content Generation in Tuning-free Long Video Diffusion (2025) · cites this paper as Everaert et al. 2024
Sections 4.2.2 and 5.3 link low-frequency components of Stable Diffusion latents to generated-image brightness and colour variation across image areas. Section 5.4 shows that selecting those components can bias brightness and colour, but our copy does not establish that they primarily correspond to layout or overall structure in pixel space. The passages do not give page numbers. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 15,880 · see citing works
An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion
Gal, R., Alaluf, Y., Atzmon, Y., Patashnik, O., Bermano, A. H., Chechik, G., & Cohen-Or, D. (2022) · arXiv · doi:10.48550/arxiv.2208.01618 · Cited by 611 · see citing works
Progressive Distillation for Fast Sampling of Diffusion Models
Salimans, T., & Ho, J. (2022) · arXiv · doi:10.48550/arxiv.2202.00512 · Cited by 287 · see citing works
Flagged first, then by citations
Research works printed but not linked: 29
- Geoffrey J Burton and Ian R Moorhead. Color and spatial structure in natural scenes. Applied optics, 26(1):157-170, 1987. 4
- Majed El Helou, Ruofan Zhou, and Sabine Süsstrunk. Stochastic frequency masking to improve super-resolution and denoising networks. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XVI 16, pages 749-766. Springer, 2020. 4
- Martin Nicolas Everaert, Marco Bocchio, Sami Arpa, Sabine Süsstrunk, and Radhakrishna Achanta. Diffusion in Style. In Proceedings of the IEEE International Conference on Com- puter Vision, 2023. 3, 5, 7
- David J Field. Relations between the statistics of natural images and the response properties of cortical cells. Journal of the Optical Society of America A, 4(12):2379-2394, 1987.
- Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. Advances in neural information processing sys- tems, 30, 2017. 6
- Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020. 1, 2, 3
- Jonathan Ho and Tim Salimans. Classifier-Free Diffusion Guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021. 6
- Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations, 2021. 6
- Diederik P Kingma and Max Welling. Auto-encoding varia- tional bayes. arXiv preprint arXiv:1312.6114, 2013. 4 · arXiv
- Mingxiao Li, Tingyu Qu, Wei Sun, and Marie-Francine Moens. Alleviating Exposure Bias in Diffusion Models through Sampling with Shifted Time Steps. arXiv preprint arXiv:2305.15583, 2023. 1, 3, 5 · arXiv
- Shanchuan Lin, Bingchen Liu, Jiashi Li, and Xiao Yang. Common Diffusion Noise Schedules and Sample Steps are Flawed. arXiv preprint arXiv:2305.08891, 2023. 1, 2, 3, 4, 5, 6, 7, 8 · arXiv
- Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo Nu- merical Methods for Diffusion Models on Manifolds. In In- ternational Conference on Learning Representations, 2022. 3, 6
- Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. SDEdit: Guided Im- age Synthesis and Editing with Stochastic Differential Equa- tions. In International Conference on Learning Representa- tions, 2021. 5
- Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162-8171. PMLR, 2021. 3
- Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mcgrew, Ilya Sutskever, and Mark Chen. GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models. In International Conference on Machine Learning, pages 16784-16804. PMLR, 2021. 3
- Mang Ning, Enver Sangineto, Angelo Porrello, Simone Calderara, and Rita Cucchiara. Input Perturbation Reduces Exposure Bias in Diffusion Models. In Proceedings of the 40th International Conference on Machine Learning, pages 26245-26265, 2023. 1, 3, 5
- Zhihong Pan, Xin Zhou, and Hao Tian. Arbitrary style guid- ance for enhanced diffusion-based text-to-image generation. In Proceedings of the IEEE/CVF Winter Conference on Ap- plications of Computer Vision, pages 4461-4471, 2022. 7
- Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, pages 8748-8763. PMLR, 2021. 6
- Marc'Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. Sequence level training with recurrent neural networks. In 4th International Conference on Learn- ing Representations, ICLR 2016, 2015. 3
- Daniel L Ruderman and William Bialek. Statistics of nat- ural images: Scaling in the woods. Physical review letters, 73(6):814, 1994. 4
- Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22500- 22510, 2022. 6, 7
- Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35:36479-36494, 2022. 6, 8
- Florian Schmidt. Generalization in Generation: A closer look at Exposure Bias. EMNLP-IJCNLP 2019, page 157, 2019. 3
- Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing Diffusion Implicit Models. In International Conference on Learning Representations, 2021. 3
- Gabriela Ben Melech Stan, Diana Wofk, Scottie Fox, Alex Redden, Will Saxton, Jean Yu, Estelle Aflalo, Shao-Yen Tseng, Fabio Nonato, Matthias Muller, et al. LDM3D: Latent Diffusion Model for 3D. arXiv preprint arXiv:2305.10853, 2023. 3, 4 · arXiv
- Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception archi- tecture for computer vision. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, pages 2818-2826, 2015. 6
- David J Tolhurst, Yoav Tadmor, and Tang Chao. Amplitude spectra of natural images. Ophthalmic and Physiological Optics, 12(2):229-232, 1992. 4
- Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Er- mon, and Jian Tang. GeoDiff: A Geometric Diffusion Model for Molecular Conformation Generation. In International Conference on Learning Representations, 2022. 3
- Yufan Zhou, Bingchen Liu, Yizhe Zhu, Xiao Yang, Changyou Chen, and Jinhui Xu. Shifted Diffusion for Text- to-image Generation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 10157-10166, 2023.
Cited by
Sorted by citations
Ouroboros-Diffusion: Exploring Consistent Content Generation in Tuning-free Long Video Diffusion
Chen, J., Long, F., An, J., Qiu, Z., Yao, T., Luo, J., & Mei, T. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i2.32205 · Cited by 5 · see citing works
- Authors
- Everaert, M. N., Fitsios, A., Bocchio, M., Arpa, S., Süsstrunk, S., & Achanta, R.
- Affiliations
- École Polytechnique Fédérale de Lausanne (CH)
- Largo.ai,Lausanne,Switzerland
- EPFL,School of Computer and Communication Sciences,Switzerland
- Venue
- 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
- Publisher
- IEEE
- Published
- 2024-01-03
- Pages
- 4013-4022
- ISSN
- 2472-6737 (ISSN-L)
- OpenAlex
- W4394625549
- Cited by
- 13
- References
- 3
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 8
- FWCI
- 2.42
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Winter Conference on Applications of Computer Vision (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- Funders
- Innosuisse - Schweizerische Agentur für Innovationsförderung (48552.1)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Signal Denoising Methods
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- diffusion models, inference bias, brightness control, color control
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
