|

M8.7 - Quantization-Aware Training and Low-Bit Optimization

A derivation- and implementation-oriented treatment of quantization-aware training: fake-quant arithmetic, straight-through gradients, learned step sizes and clipping, observer schedules, LLM-QAT data-free distillation, weight/activation/KV insertion sites, efficient QAT variants, distributed training state, and export parity.