#multi-head-attention • Tags • Transformer System

Press Esc to close

1 post · Transformer System

Tag: #multi-head-attention

M3.2 — Multi-Head and Cross-Attention: Projection, Layout, and Variants

🗓 2026-07-10 • Transformer System • ⏱ 94 min read

A code-, shape-, and systems-oriented study of packed multi-head attention, target-by-source cross-attention, head composition, conditional head routing, concatenation, and output projection.

#numinens #transformer #attention #multi-head-attention #cross-attention #mha

Read →

Su	Mo	Tu	We	Th	Fr	Sa
27	28	29	30	1	2	3
4	5	6	7	8	9	10
11	12	13	14	15	16	17
18	19	20	21	22	23	24
25	26	27	28	29	30	31

| #multi-head-attention

Tag: #multi-head-attention

M3.2 — Multi-Head and Cross-Attention: Projection, Layout, and Variants