|

M5.7 - MQA and GQA: KV-Head Sharing, Cache Bytes, and Grouped Decode Kernels

A quantitative reconstruction of multi-query and grouped-query attention: KV-head ownership, checkpoint conversion, cache bytes, logical broadcast, grouped decode reuse, tensor parallelism, and failure diagnosis.