Agile-Quant: Activation-Guided Quantization for Faster Inference of LLMs on the Edge

Shen, Xuan, Dong, Peiyan, Lu, Lei · arXiv · 2023