M3.6 — Dense and Gated MLPs: Channel Mixing, Training, and Runtime
A derivation- and implementation-oriented study of Transformer MLPs: position-wise channel mixing, activation and GLU variants, SwiGLU width matching, backward flow, fused execution, knowledge interpretations, and memory alternatives.