A quantitative reconstruction of multi-query and grouped-query attention: KV-head ownership, checkpoint conversion, cache bytes, logical broadcast, grouped decode reuse, tensor parallelism, and failure diagnosis.
A code-, shape-, and systems-oriented study of packed multi-head attention, target-by-source cross-attention, head composition, conditional head routing, concatenation, and output projection.
A code- and shape-oriented study note on scaled dot-product attention: Q/K/V, score tensors, finite bias, visibility masks, softmax over keys, dropout, and weighted value mixing.
A tensor- and code-oriented reconstruction of FlashAttention-1: online softmax, exact tiled attention, HBM traffic, backward recomputation, masks, and backend evidence.
Sequence modeling from information dependencies to executable Transformer architectures: recurrence, convolution, attention, training, and inference.