A tensor-, algebra-, and kernel-level reconstruction of Multi-Head Latent Attention: latent KV cache state, decoupled RoPE, weight absorption, DeepSeek reference paths, FlashMLA scheduling, FP8 cache layout, sparse variants, and parity diagnosis.
A rigorous study of why low-rank adaptation can work and where it fails: intrinsic-dimension measurement, matrix manifolds, SVD reconstruction, factor redundancy, initialization gradients, NTK preservation, expressive-power bounds, feature learning, and forgetting.
A derivation- and implementation-oriented study of LoRA: low-rank update geometry, tensor shapes, forward and backward passes, initialization, scaling, optimizer state, target-module selection, and the limits of the low-rank assumption.