Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
PreprintCited by 43Advanced Neural Network ApplicationsAdversarial Robustness in Machine LearningGenerative Adversarial Networks and Image Synthesis

The DOI matches its DataCite record; the references it cites are not checked yet.

PTQD: Accurate Post-Training Quantization for Diffusion Models

Open access
He, Yefei, Liu, Luping, Liu, Jing, Wu, Weijia, Zhou, Hong, Zhuang, Bohan·2023·arXiv
DOI 10.48550/arxiv.2305.10657arXiv 2305.10657
Open this paper in CoChat — free accountOpen PDF

Abstract

Diffusion models have recently dominated image synthesis tasks. However, the iterative denoising process is expensive in computations at inference time, making diffusion models less practical for low-latency and scalable real-world applications. Post-training quantization (PTQ) of diffusion models can significantly reduce the model size and accelerate the sampling process without re-training. Nonetheless, applying existing PTQ methods directly to low-bit diffusion models can significantly impair the quality of generated samples. Specifically, for each denoising step, quantization noise leads to deviations in the estimated mean and mismatches with the predetermined variance schedule. As the sampling process proceeds, the quantization noise may accumulate, resulting in a low signal-to-noise ratio (SNR) during the later denoising steps. To address these challenges, we propose a unified formulation for the quantization noise and diffusion perturbed noise in the quantized denoising process. Specifically, we first disentangle the quantization noise into its correlated and residual uncorrelated parts regarding its full-precision counterpart. The correlated part can be easily corrected by estimating the correlation coefficient. For the uncorrelated part, we subtract the bias from the quantized results to correct the mean deviation and calibrate the denoising variance schedule to absorb the excess variance resulting from quantization. Moreover, we introduce a mixed-precision scheme for selecting the optimal bitwidth for each denoising step. Extensive experiments demonstrate that our method outperforms previous post-training quantized diffusion models, with only a 0.06 increase in FID score compared to full-precision LDM-4 on ImageNet 256x256, while saving 19.9x bit operations. Code is available at https://github.com/ziplab/PTQD.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2305.10657

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

The citation lists haven’t been fetched yet

Cited by 43

Research works printed but not linked: 65

  1. [1][1] F. Bao, C. Li, J. Sun, J. Zhu, and B. Zhang. Estimating the optimal covariance with imperfect mean in diffusion probabilistic models. In ICML , 2022. -
  2. [2][2] F. Bao, C. Li, J. Zhu, and B. Zhang. Analytic-dpm: an analytic estimate of the optimal reverse variance in diffusion probabilistic models. In ICLR , 2022. -
  3. [4][4] Y. Cai, Z. Yao, Z. Dong, A. Gholami, M. W. Mahoney, and K. Keutzer. Zeroq: A novel zero shot quantization framework. In CVPR , 2020. -
  4. [5][5] N. Chen, Y. Zhang, H. Zen, R. J. Weiss, M. Norouzi, and W. Chan. Wavegrad: Estimating gradients for waveform generation. In ICLR , 2021. -
  5. [6][6] W. Chen, P. Wang, and J. Cheng. Towards mixed-precision quantization of neural networks via constrained optimization. In ICCV , 2021. -
  6. [7][7] H. Chung, B. Sim, and J. C. Ye. Come-closer-diffuse-faster: Accelerating conditional diffusion models for inverse problems through stochastic contraction. In CVPR , 2022. -
  7. [9][9] R. D'AGOSTINO and E. S. Pearson. Tests for departure from normality. empirical results for the distributions of b 2 and √ b . Biometrika , 60(3):613-622, 1973. -
  8. [10][10] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR , 2009. -
  9. [11][11] P. Dhariwal and A. Q. Nichol. Diffusion models beat gans on image synthesis. In NeurIPS , pages 8780-8794, 2021. -
  10. [12][12] R. DIAgostino. An omnibus test of normality for moderate and large sample sizes. Biometrika , 58(34):1348, 1971. -
  11. [13][13] Z. Dong, Z. Yao, A. Gholami, M. W. Mahoney, and K. Keutzer. Hawq: Hessian aware quantization of neural networks with mixed-precision. In ICCV , 2019. -
  12. [14][14] G. Franzese, S. Rossi, L. Yang, A. Finamore, D. Rossi, M. Filippone, and P. Michiardi. How much is enough? a study on diffusion times in score-based generative models. arXiv preprint arXiv:2206.05173 , 2022. - · arXiv
  13. [15][15] G. Giannone, D. Nielsen, and O. Winther. Few-shot diffusion models. arXiv preprint arXiv:2205.15463 , 2022. - · arXiv
  14. [16][16] R. Gong, X. Liu, S. Jiang, T. Li, P. Hu, J. Lin, F. Yu, and J. Yan. Differentiable soft quantization: Bridging full-precision and low-bit neural networks. In ICCV , 2019. -
  15. [17][17] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial networks. Communications of the ACM , 63(11):139-144, 2020. -
  16. [18][18] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In NeurIPS , volume 30, 2017. -
  17. [19][19] J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. In NeurIPS , 2020. -
  18. [20][20] J. Ho, T. Salimans, A. A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet. Video diffusion models. In NeurIPS , 2022. -
  19. [21][21] X. Huang, Z. Shen, S. Li, Z. Liu, H. Xianghong, J. Wicaksana, E. Xing, and K.-T. Cheng. Sdq: Stochastic differentiable quantization with mixed precision. In ICML , 2022. -
  20. [22][22] I. Hubara, Y. Nahshan, Y. Hanani, R. Banner, and D. Soudry. Improving post training neural quantization: Layer-wise calibration and integer programming. arXiv preprint arXiv:2006.10518 , 2020. - · arXiv
  21. [23][23] B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In CVPR , 2018. -
  22. [24][24] B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In CVPR , 2018. -
  23. [25][25] A. Jolicoeur-Martineau, K. Li, R. Piché-Taillefer, T. Kachman, and I. Mitliagkas. Gotta go fast when generating data with score-based models. arXiv preprint arXiv:2105.14080 , 2021. - · arXiv
  24. [26][26] T. Karras, M. Aittala, T. Aila, and S. Laine. Elucidating the design space of diffusion-based generative models. In NeurIPS , 2022. -
  25. [28][28] B. Kim and J. C. Ye. Denoising mcmc for accelerating diffusion-based generative models. In ICML , 2023. -
  26. [29][29] D. Kingma, T. Salimans, B. Poole, and J. Ho. Variational diffusion models. In NeurIPS , 2021. -
  27. [30][30] D. P. Kingma and M. Welling. Auto-encoding variational bayes. In ICLR , 2014. -
  28. [31][31] Z. Kong and W. Ping. On fast sampling of diffusion probabilistic models. In ICLR , 2023. -
  29. [32][32] M. W. Y. Lam, J. Wang, D. Su, and D. Yu. BDDM: bilateral denoising diffusion models for fast and high-quality speech synthesis. In ICLR , 2022. -
  30. [33][33] A. Levkovitch, E. Nachmani, and L. Wolf. Zero-shot voice conditioning for denoising diffusion TTS models. In Interspeech , pages 2983-2987, 2022. -
  31. [34][34] X. Li, Y. Liu, L. Lian, H. Yang, Z. Dong, D. Kang, S. Zhang, and K. Keutzer. Q-diffusion: Quantizing diffusion models. In ICCV , 2023. -
  32. [35][35] Y. Li, R. Gong, X. Tan, Y. Yang, P. Hu, Q. Zhang, F. Yu, W. Wang, and S. Gu. BRECQ: pushing the limit of post-training quantization by block reconstruction. In ICLR , 2021. -
  33. [36][36] Y. Lin, T. Zhang, P. Sun, Z. Li, and S. Zhou. Fq-vit: Post-training quantization for fully quantized vision transformer. In IJCAI , 2022. -
  34. [37][37] L. Liu, Y. Ren, Z. Lin, and Z. Zhao. Pseudo numerical methods for diffusion models on manifolds. In ICLR , 2022. -
  35. [38][38] C. Louizos, M. Reisser, T. Blankevoort, E. Gavves, and M. Welling. Relaxed quantization for discretized neural networks. In ICLR , 2019. -
  36. [39][39] C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, and J. Zhu. Dpm-solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps. In NeurIPS , 2022. -
  37. [40][40] C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, and J. Zhu. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arXiv:2211.01095 , 2022. - · arXiv
  38. [41][41] E. Luhman and T. Luhman. Knowledge distillation in iterative generative models for improved sampling speed. arXiv preprint arXiv:2101.02388 , 2021. - · arXiv
  39. [42][42] S. Luo and W. Hu. Diffusion probabilistic models for 3d point cloud generation. In CVPR , 2021. -
  40. [43][43] Z. Lyu, X. Xu, C. Yang, D. Lin, and B. Dai. Accelerating diffusion models via early stop of the diffusion process. arXiv preprint arXiv:2205.12524 , 2022. - · arXiv
  41. [44][44] M. Nagel, R. A. Amjad, M. van Baalen, C. Louizos, and T. Blankevoort. Up or down? adaptive rounding for post-training quantization. In ICML , volume 119, pages 7197-7206, 2020. -
  42. [45][45] M. Nagel, M. van Baalen, T. Blankevoort, and M. Welling. Data-free quantization through weight equalization and bias correction. In ICCV , pages 1325-1334, 2019. -
  43. [46][46] C. Nash, J. Menick, S. Dieleman, and P. W. Battaglia. Generating images with sparse representations. In ICML , volume 139, pages 7958-7968, 2021. -
  44. [47][47] A. Q. Nichol and P. Dhariwal. Improved denoising diffusion probabilistic models. In ICML , 2021. -
  45. [49][49] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In CVPR , pages 10674-10685, 2022. -
  46. [50][50] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. Improved techniques for training gans. In NeurIPS , volume 29, 2016. -
  47. [51][51] T. Salimans and J. Ho. Progressive distillation for fast sampling of diffusion models. In ICLR , 2022. -
  48. [52][52] Y. Shang, Z. Yuan, B. Xie, B. Wu, and Y. Yan. Post-training quantization on diffusion models. arXiv preprint arXiv:2211.15736 , 2022. - · arXiv
  49. [53][53] C. Shi, S. Luo, M. Xu, and J. Tang. Learning gradient fields for molecular conformation generation. In ICML , 2021. -
  50. [54][54] J. Song, C. Meng, and S. Ermon. Denoising diffusion implicit models. In ICLR , 2021. -
  51. [55][55] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. In ICLR , 2021. -
  52. [56][56] P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, et al. Scipy 1.0: fundamental algorithms for scientific computing in python. Nature methods , 17(3):261-272, 2020. -
  53. [57][57] V. Voleti, A. Jolicoeur-Martineau, and C. Pal. MCVD - masked conditional video diffusion for prediction, generation, and interpolation. In NeurIPS , 2022. -
  54. [58][58] P. Wang, Q. Chen, X. He, and J. Cheng. Towards accurate post-training network quantization via bit-split and stitching. In ICML , 2020. -
  55. [59][59] Y. Wang, Y. Lu, and T. Blankevoort. Differentiable joint pruning and quantization for hardware efficiency. In ECCV , 2020. -
  56. [60][60] D. Watson, W. Chan, J. Ho, and M. Norouzi. Learning fast samplers for diffusion models by differentiating through sample quality. In ICLR , 2022. -
  57. [61][61] D. Watson, J. Ho, M. Norouzi, and W. Chan. Learning to efficiently sample from diffusion probabilistic models. arXiv preprint arXiv:2106.03802 , 2021. - · arXiv
  58. [62][62] X. Wei, R. Gong, Y. Li, X. Liu, and F. Yu. Qdrop: Randomly dropping quantization for extremely low-bit post-training quantization. In ICLR , 2022. -
  59. [63][63] W. Wu, Y. Zhao, M. Z. Shou, H. Zhou, and C. Shen. Diffumask: Synthesizing images with pixel-level annotations for semantic segmentation using diffusion models. arXiv preprint arXiv:2303.11681 , 2023. - · arXiv
  60. [64][64] H. Yang, L. Duan, Y. Chen, and H. Li. BSQ: exploring bit-level sparsity for mixed-precision neural network quantization. In ICLR , 2021. -
  61. [65][65] F. Yu, A. Seff, Y. Zhang, S. Song, T. Funkhouser, and J. Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365 , 2015. - · arXiv
  62. [66][66] L. Zhang, Y. He, Z. Lou, X. Ye, Y. Wang, and H. Zhou. Root quantization: a self-adaptive supplement ste. Applied Intelligence , 53(6):6266-6275, 2023. -
  63. [67][67] Q. Zhang and Y. Chen. Fast sampling of diffusion models with exponential integrator. arXiv preprint arXiv:2204.13902 , 2022. - · arXiv
  64. [68][68] Q. Zhang, M. Tao, and Y. Chen. gddim: Generalized denoising diffusion implicit models. In ICLR , 2023. -
  65. [70][70] B. Zhuang, C. Shen, M. Tan, L. Liu, and I. Reid. Towards effective low-bitwidth convolutional neural networks. In CVPR , 2018.

Metadata

Authors
He, Y., Liu, L., Liu, J., Wu, W., Zhou, H., & Zhuang, B.
Venue
arXiv
Publisher
arXiv
Published
2023
Pages
13237-13249
ISSN
1049-5258 (ISSN-L)
DOI
10.48550/arxiv.2305.10657
arXiv
2305.10657
OpenAlex
W7133236699
Cited by
43
References
0
Citations by year
2026: 13 · 2025: 22 · 2024: 7 · 2023: 1
FWCI
4.61
Type
conference paper
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Monash University Research Portal (Monash University) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Funders
National Key Research and Development Program of China (2022YFC3602601)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Keywords
post-training quantization, diffusion models, model compression, quantization-aware training, neural network quantization

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: PTQD: Accurate Post-Training Quantization for Diffusion Models. https://app.cochat.ai/papers/10.48550/arxiv.2305.10657
The paper
He, Y., Liu, L., Liu, J., Wu, W., Zhou, H., & Zhuang, B. (2023). PTQD: Accurate Post-Training Quantization for Diffusion Models. arXiv. https://doi.org/10.48550/arxiv.2305.10657