A quantitative reconstruction of multi-query and grouped-query attention: KV-head ownership, checkpoint conversion, cache bytes, logical broadcast, grouped decode reuse, tensor parallelism, and failure diagnosis.
A tensor-, algebra-, and kernel-level reconstruction of Multi-Head Latent Attention: latent KV cache state, decoupled RoPE, weight absorption, DeepSeek reference paths, FlashMLA scheduling, FP8 cache layout, sparse variants, and parity diagnosis.
A distribution- and serving-state-oriented study note on speculative decoding, draft models, target verification, exact speculative sampling, correction distributions, acceptance rate, speedup ledgers, and KV rollback.
A mechanism-oriented study note on Medusa decoding heads, same-model candidate generation, candidate trees, tree attention masks, typical acceptance, Medusa-1, Medusa-2, and branch KV state.
A mechanism-oriented study note on Lookahead decoding, Jacobi iterations, 2D windows, n-gram pools, lookahead and verification branches, exactness, masks, and KV state.
A serving-oriented study note on KV cache lifecycle, prefill, decode, TTFT, TPOT, memory admission, and why autoregressive inference turns attention state into a scheduler problem.
A systems-oriented study note on prefill/decode interference, chunked prefill, disaggregated serving, KV transfer, placement, routing, goodput, and exact cache handoff contracts.
A code- and shape-oriented study note on Transformer cost: parameters, activations, FLOPs, memory traffic, prefill, decode, and why FLOPs alone do not predict runtime.
A kernel-oriented reconstruction of FlashAttention-2 work partitioning, parallelism across sequence rows, prefill/decode shape changes, and the boundary where decode requires a different schedule.
A derivation- and implementation-oriented study of one-token decode, Flash-Decoding, exact partial-softmax reduction, FlashMask, GQA-aware cache access, SDPA dispatch, and the FA2-to-FA3 kernel boundary.