A tensor- and state-machine study of multimodal placeholder expansion, sequence packing, attention and loss masks, training stages, prefill/decode execution, media caching, batching, and safe reuse.
A derivation- and implementation-oriented study of exact KV offload, compute-near-data attention, hierarchical residency, query-dependent retrieval, recallable memory, consistency, fallback, and serving evaluation.
A quantitative reconstruction of multi-query and grouped-query attention: KV-head ownership, checkpoint conversion, cache bytes, logical broadcast, grouped decode reuse, tensor parallelism, and failure diagnosis.
A tensor-, algebra-, and kernel-level reconstruction of Multi-Head Latent Attention: latent KV cache state, decoupled RoPE, weight absorption, DeepSeek reference paths, FlashMLA scheduling, FP8 cache layout, sparse variants, and parity diagnosis.
A tensor-level study of routing outside FFN experts: Mixture of Attention Heads, SwitchHead, Mixture-of-Depths, causal route prediction, per-layer KV consequences, MoD+MoE composition, and staged multimodal sparse training.
A derivation-, state-, scheduler-, and operations-level treatment of continuous-batch speculative decoding, adaptive draft budgets, goodput control, target and draft cache synchronization, evaluation, canary release, and rollback.
A derivation- and implementation-oriented treatment of request-dependent quantization: dynamic activation scales, integer GEMM epilogues, attention scale placement, persistent KV-cache rate and metadata, streaming append state, KIVI, KVQuant, PrefixQuant, QServe W4A8KV4, production FP8 cache formats, and serving correctness.
A derivation- and implementation-oriented treatment of quantization-aware training: fake-quant arithmetic, straight-through gradients, learned step sizes and clipping, observer schedules, LLM-QAT data-free distillation, weight/activation/KV insertion sites, efficient QAT variants, distributed training state, and export parity.
A systems study of serving many LoRA adapters over one base model: heterogeneous batching, segmented kernels, paging, residency, merge and hotswap semantics, request/KV compatibility, scheduling, observability, and failure recovery.
A formula-, tensor-, code-, and runtime-level study of DeepSeek-V3 multi-token prediction modules, shared embeddings and heads, teacher forcing, released weight structure, KV ownership, and speculative serving.
A distribution- and serving-state-oriented study note on speculative decoding, draft models, target verification, exact speculative sampling, correction distributions, acceptance rate, speedup ledgers, and KV rollback.
A mechanism-oriented study note on Medusa decoding heads, same-model candidate generation, candidate trees, tree attention masks, typical acceptance, Medusa-1, Medusa-2, and branch KV state.
A serving-oriented study note on KV cache lifecycle, prefill, decode, TTFT, TPOT, memory admission, and why autoregressive inference turns attention state into a scheduler problem.
A serving-scheduler study note on static batching failure, iteration-level scheduling, continuous batching, chunked prefill, TTFT, TPOT, goodput, and KV-aware admission control.
A derivation- and implementation-oriented study of physical KV layout, PagedAttention, vAttention, exact prefix identity, RadixAttention, ownership, and execution reuse.
A systems-oriented study of long-context KV pressure, sliding windows, attention sinks, H2O, SnapKV, PyramidKV, PyramidInfer, layer/head budgets, and the accuracy contracts behind lossy cache policies.
A systems-oriented study note on prefill/decode interference, chunked prefill, disaggregated serving, KV transfer, placement, routing, goodput, and exact cache handoff contracts.
A code- and shape-oriented study note on Transformer cost: parameters, activations, FLOPs, memory traffic, prefill, decode, and why FLOPs alone do not predict runtime.
A kernel-oriented reconstruction of FlashAttention-2 work partitioning, parallelism across sequence rows, prefill/decode shape changes, and the boundary where decode requires a different schedule.
A derivation- and implementation-oriented study of one-token decode, Flash-Decoding, exact partial-softmax reduction, FlashMask, GQA-aware cache access, SDPA dispatch, and the FA2-to-FA3 kernel boundary.
A derivation- and implementation-oriented study of one autoregressive request: prompt state, prefill/decode, last-row logits, token selection, KV append timing, positions, stopping, beam reorder, streaming, and cache equivalence.