|

M8.6 - Live-Data Quantization: Activations and KV Cache

A derivation- and implementation-oriented treatment of request-dependent quantization: dynamic activation scales, integer GEMM epilogues, attention scale placement, persistent KV-cache rate and metadata, streaming append state, KIVI, KVQuant, PrefixQuant, QServe W4A8KV4, production FP8 cache formats, and serving correctness.