qat question?

[coolKeen]

Hi, thanks for your great job!!!
Does llm-compressor support loading ptq weights to continue finetuing weights (i.e. qat) without changing quantization parameters (e.g. scale/zero point)?

[brian-dellabetta]

Hi @coolKeen , I don't think you can continue training after quantization (except maybe the case of fp8 dynamic quantization). weights are always integers when a scale & zero_point are involved, and back-prop isn't possible with integer weights. If you want to quantize to fp8 and then train, some frameworks might allow for that, but it is not a focus for llm-compressor