M4.3 - IO-Aware Attention and FlashAttention-1: Online Softmax, Tiling, and Exactness
A tensor- and code-oriented reconstruction of FlashAttention-1: online softmax, exact tiled attention, HBM traffic, backward recomputation, masks, and backend evidence.