A quantitative reconstruction of FlashAttention-3 and FlashAttention-4: Hopper TMA/WGMMA pipelines, Blackwell TMEM and 2-CTA MMA, softmax bottlenecks, FP8 accuracy, backward scheduling, and benchmark evidence.
A derivation-, state-, scheduler-, and operations-level treatment of continuous-batch speculative decoding, adaptive draft budgets, goodput control, target and draft cache synchronization, evaluation, canary release, and rollback.
A systems study of serving many LoRA adapters over one base model: heterogeneous batching, segmented kernels, paging, residency, merge and hotswap semantics, request/KV compatibility, scheduling, observability, and failure recovery.
A serving-oriented study note on KV cache lifecycle, prefill, decode, TTFT, TPOT, memory admission, and why autoregressive inference turns attention state into a scheduler problem.
A systems-oriented study note on prefill/decode interference, chunked prefill, disaggregated serving, KV transfer, placement, routing, goodput, and exact cache handoff contracts.