A tensor- and state-machine study of multimodal placeholder expansion, sequence packing, attention and loss masks, training stages, prefill/decode execution, media caching, batching, and safe reuse.
A quantitative reconstruction of multi-query and grouped-query attention: KV-head ownership, checkpoint conversion, cache bytes, logical broadcast, grouped decode reuse, tensor parallelism, and failure diagnosis.
A tensor-, algebra-, and kernel-level reconstruction of Multi-Head Latent Attention: latent KV cache state, decoupled RoPE, weight absorption, DeepSeek reference paths, FlashMLA scheduling, FP8 cache layout, sparse variants, and parity diagnosis.
A derivation- and implementation-oriented treatment of request-dependent quantization: dynamic activation scales, integer GEMM epilogues, attention scale placement, persistent KV-cache rate and metadata, streaming append state, KIVI, KVQuant, PrefixQuant, QServe W4A8KV4, production FP8 cache formats, and serving correctness.
A serving-oriented study note on KV cache lifecycle, prefill, decode, TTFT, TPOT, memory admission, and why autoregressive inference turns attention state into a scheduler problem.
A serving-scheduler study note on static batching failure, iteration-level scheduling, continuous batching, chunked prefill, TTFT, TPOT, goodput, and KV-aware admission control.
A derivation- and implementation-oriented study of physical KV layout, PagedAttention, vAttention, exact prefix identity, RadixAttention, ownership, and execution reuse.
A systems-oriented study note on prefill/decode interference, chunked prefill, disaggregated serving, KV transfer, placement, routing, goodput, and exact cache handoff contracts.
A kernel-oriented reconstruction of FlashAttention-2 work partitioning, parallelism across sequence rows, prefill/decode shape changes, and the boundary where decode requires a different schedule.