A tensor-, algebra-, and kernel-level reconstruction of Multi-Head Latent Attention: latent KV cache state, decoupled RoPE, weight absorption, DeepSeek reference paths, FlashMLA scheduling, FP8 cache layout, sparse variants, and parity diagnosis.
A formula-, tensor-, code-, and runtime-level study of DeepSeek-V3 multi-token prediction modules, shared embeddings and heads, teacher forcing, released weight structure, KV ownership, and speculative serving.
A systems-oriented study note on scaled Mixture-of-Experts routing, load balancing, expert parallelism, all-to-all communication, shared experts, fine-grained experts, device-limited routing, and DeepSeek-style MoE.