M5.8 - Multi-Head Latent Attention: Latent KV State, Weight Absorption, and FlashMLA

M5.8 - Multi-Head Latent Attention: Latent KV State, Weight Absorption, and FlashMLA

0x00 Overview And Background

M5.7 reduced K/V cache by changing the number of K/V heads:

MHA:
    cache H_q K heads and H_q V heads

GQA:
    cache fewer H_kv heads and share each across a query-head group

MQA:
    cache one K head and one V head

Multi-Head Latent Attention changes a deeper object. Instead of deciding only how many full K/V heads exist, it asks whether historical content must be stored as full per-head K/V at all.

ordinary cache row:
    full projected K and V features

MLA cache row:
    compressed latent K/V content
    plus the positional key component that cannot be absorbed statically

DeepSeek-V2’s full architecture figure locates that change inside the block. The lower panel is the relevant one here: the hidden row is down-projected into a latent content object and a RoPE key path before multi-head attention, and the hatched latent/positional state is what remains cached during inference.

Source: DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model, Figure 2. M5.8 imports the lower MLA panel; the upper DeepSeekMoE panel belongs to M6. The hatched objects identify inference-cached state in the paper’s architecture.

That placement view and the next cache-comparison view answer different questions: Figure 2 shows where MLA enters the model path; Figure 3 shows how its persistent object differs from MHA, GQA, and MQA.

Source: DeepSeek-V2, Figure 3. Hatched objects are cached for inference. The figure introduces the representation change; the exact latent, RoPE, and kernel contracts are derived below.

This is not ordinary low-rank compression applied after serving. The model is trained with a joint latent K/V path. Efficient inference then uses matrix associativity to move key expansion to the current query and value expansion to the small post-attention result.

The full mechanism crosses five layers:

1. model: down-project hidden rows into a latent K/V representation;
2. position: keep a separate RoPE key path;
3. algebra: absorb key and value up-projections into other contractions;
4. state: cache latent plus RoPE rows, not reconstructed per-head K/V;
5. kernel: read that state directly with a layout and schedule built for it.

The thesis:

MLA reduces serving state only when the runtime preserves the latent
representation across cache publication, addressing, score construction,
softmax/value accumulation, and output projection.

A checkpoint can be MLA while a naive runtime reconstructs MHA-sized history
and erases much of the intended serving benefit.

Historical lineage: from fewer heads to a different cached object

The path to MLA is easier to understand as a sequence of questions rather than as a list of attention acronyms.

MHA:
    can every query head retain its own K/V representation?

MQA and GQA:
    can several query heads share a smaller number of full K/V heads?

low-rank K/V proposals:
    can projected K and V live in lower-dimensional subspaces?

MLA:
    can K and V share one learned latent content state, while the runtime
    moves the expansion matrices out of the history-length contractions?

The first transition changes head multiplicity. The last transition changes the representation stored for every historical token. This distinction is why MLA belongs after M5.7 but cannot be reduced to “MQA with a larger head.”

Source-local figure from A28, derived from DeepSeek-V2. The Chinese annotations mark compression and copying. The semantic point is independent of those labels: MQA/GQA reduce the number of full K/V streams, whereas MLA stores a joint latent content row plus a positional row.

DeepSeek-V2 introduced the native MLA architecture in 2024. Later work asks a different question: whether an already-trained MHA or GQA checkpoint can be converted into an MLA-like factorization. Those conversion methods are not retroactive explanations of native MLA. They have additional approximation, initialization, and adaptation contracts, treated in 0x0D.

Owner question

How can a model retain multi-head attention capacity while making historical K/V state latent, and which exact algebra, cache schema, and runtime path realize that reduction?

The answer has four independent proof obligations:

1. representation: the checkpoint must produce a joint latent content row;
2. algebra: the up-projections must be reassociated without changing dense attention semantics;
3. state: prefill and append must publish the latent and rotary rows rather than reconstructed head-expanded K/V;
4. execution: the selected backend must consume that schema directly.

A failure in any one obligation can leave the model mathematically MLA while the deployed system pays MHA-like traffic.

Immutable contract manifest

Before deriving anything, freeze the objects being compared:

This is not documentation overhead. MLA has several shape-compatible failure modes: a 576-wide row can mean BF16 [512 latent | 64 RoPE], a packed FP8 row can occupy 656 bytes, and an expanded implementation can expose a head-shaped tensor without changing the checkpoint. Width alone is not a schema.

Ownership boundary

M5.8 owns the latent cache object, absorption algebra, reference execution paths, FlashMLA ABI, dense/sparse and BF16/FP8 cache formats, hardware schedule, parallel ownership, and parity harness.

It does not rederive RoPE, PagedAttention, the general online-softmax merge, Split-K decode reduction, or FP8 numerical formats from first principles. Those are owned by M3.4, M5.3, M4.3, M4.5, and M8.6, respectively. Here they are imported only where they become part of the MLA execution contract.

The complete boundary is:

Claim-status contract

The rest of the article follows that status discipline. Equations derived here are not presented as paper quotations; paper and runtime measurements remain attached to their exact source coordinates.

0x01 Local Tensor Contract And Notation

Use:

For one hidden row h_t:

query path:
    h_t -> c_q[t] -> q_nope[t], q_rope[t]

joint K/V path:
    h_t -> c_kv[t], k_rope[t]

persistent row:
    concat(c_kv[t], k_rope[t])

The efficient persistent payload per token per layer is:

$$M_{token,layer}^{MLA}=(d_c+d_R)b.$$

For B equal-length requests:

$$M_{cache}^{MLA}=BTL(d_c+d_R)b.$$

This formula is the architecture-level payload. Physical residency can add page padding, alignment, quantization scales, block tables, replicas, workspace, and derived absorbed weights.

Checkpoint-to-tensor trace

The official DeepSeek-V3 reference implementation provides an unusually useful executable architecture manifest. At the revision pinned in the evidence bundle, the attention module declares the following source tensors:

The source-to-tensor path is:

h [B,T,d]
  |-- wq_a -> RMSNorm -> wq_b
  |      -> q [B,T,H_q,d_N+d_R]
  |      -> q_nope [B,T,H_q,d_N]
  |      -> RoPE(q_pe) [B,T,H_q,d_R]
  |
  |-- wkv_a
         -> c_raw [B,T,d_c] -> RMSNorm -> c_kv [B,T,d_c]
         -> RoPE(k_pe) [B,T,d_R]

naive path:
    wkv_b(c_kv) -> [k_nope per head | value per head]
    cache expanded k and value

absorbed path:
    cache [c_kv | k_pe]
    reshape wkv_b into head-local factors for score/output reassociation

Source-local figure from A28. It is dense and contains Chinese annotations, so the table above is the accessible contract. The figure is retained because it connects paper symbols, checkpoint names, tensor shapes, and execution edges in one artifact.

The two reference cache schemas are visible in code as well:

attn_impl = "naive":
    k_cache [B_max,T_max,H_local,d_N+d_R]
    v_cache [B_max,T_max,H_local,d_V]

attn_impl = "absorb":
    kv_cache [B_max,T_max,d_c]
    pe_cache [B_max,T_max,d_R]

This trace is more authoritative for implementation claims than an architecture picture alone. The evidence bundle pins the exact file, commit, and SHA-256 so a later source change cannot silently change the article’s ABI.

For the running DeepSeek-V2-shaped fixture, those symbolic shapes become:

The fused orders are part of identity:

wq_b per-head row order: [q_nope, q_rope]
wkv_a row order:         [c_kv_raw, k_rope_raw]
wkv_b per-head row order:[k_nope, value]
cache row order:         [normalized_c_kv, rotated_k_rope]

Canonical identity is computed from sorted semantic JSON, not file whitespace. Live request generations and allocator epochs are receipts under this identity; they do not mutate it.

def canonical_mla_id(manifest):
    semantic = {name: manifest[name] for name in (
        "architecture", "checkpoint", "position", "cache", "kernel",
        "placement", "semantic_visibility"
    )}
    payload = json.dumps(semantic, sort_keys=True, separators=(",", ":"))
    return "sha256:" + sha256(payload.encode()).hexdigest()

Checkpoint admission derives every expected shape and rejects the artifact before page allocation if a tensor, fused order, normalization stage, RoPE contract, cache field, backend semantic mode, or placement partition differs. Any such change rotates the cache namespace. A performance-only kernel update may preserve it only after parity proves unchanged cache and visibility semantics.

0x02 Reconstructing The MLA Model Path

Query projection

A conceptual DeepSeek-style query path is:

$$c_t^Q=\operatorname{RMSNorm}(W^{DQ}h_t) \in\mathbb{R}^{d_c'}.$$

The query up-projection emits all head-local non-positional and rotary features:

q_full = W_UQ(c_q)
q_full: [B, T_q, H_q, d_N + d_R]

split:
    q_nope: [B, H_q, T_q, d_N]
    q_rope: [B, H_q, T_q, d_R]

apply RoPE only to q_rope

The query latent is current-call activation state. It is not retained for every historical token.

Joint K/V projection

One down-projection emits a content latent and a positional key component:

kv_down(h_t):
    c_kv_raw:  [B, T_q, d_c]
    k_rope_raw:[B, T_q, d_R]

c_kv   = RMSNorm(c_kv_raw)
k_rope = RoPE(k_rope_raw, position=t)

The content latent can be expanded into per-head non-positional keys and values:

k_nope = W_UK(c_kv): [B, H_q, T_q, d_N]
v      = W_UV(c_kv): [B, H_q, T_q, d_V]

The shared positional key is logically visible to every query head:

logical key for head i:
    concat(k_nope[i], k_rope)

query for head i:
    concat(q_nope[i], q_rope[i])

The score separates:

$$s_{i,t,j} =q_{i,t}^{N\mathsf T}k_{i,j}^{N} +q_{i,t}^{R\mathsf T}k_j^R.$$

Renderer-safe form:

score[head i, query t, key j]
    = dot(q_nope[i,t], k_nope[i,j])
    + dot(q_rope[i,t], k_rope[j])

That decomposition explains why the cache row is not only c_kv[j]:

content score:
    can be recovered or absorbed from c_kv[j]

position score:
    still needs the position-rotated k_rope[j]

efficient persistent row:
    [c_kv[j], k_rope[j]]

Concrete DeepSeek-V2 dimensions

Using dimensions reported for DeepSeek-V2:

d        = 5120
H_q      = 128
d_c'     = 1536
d_c      = 512
d_N      = 128
d_R      = 64
d_V      = 128
L        = 60

For B=2, T_q=1, and T_k=4096:

h_new:       [2, 1, 5120]
c_q:         [2, 1, 1536]
q_nope:      [2, 128, 1, 128]
q_rope:      [2, 128, 1, 64]

c_kv_new:    [2, 1, 512]
k_rope_new:  [2, 1, 64]

latent cache:[2, 4096, 512]
RoPE cache:  [2, 4096, 64]

The current query is head-expanded. Historical content is not.

Parameter and persistent-state reconciliation

Low-rank state does not imply fewer parameters everywhere. Write each learned matrix before comparing the architecture with a conventional attention block. Ignoring biases and normalization vectors:

$$P_Q=d\,d_c'+d_c'H_q(d_N+d_R).$$

Renderer-safe form:

P_Q = d * d_c_prime + d_c_prime * H_q * (d_N + d_R)

The joint K/V path has:

$$P_{KV}=d(d_c+d_R)+d_cH_q(d_N+d_V).$$

The output projection has:

$$P_O=dH_qd_V.$$

Therefore the complete learned linear-parameter ledger for one MLA attention layer is:

$$P_{\text{attention-linear}}=P_Q+P_{KV}+P_O.$$

Renderer-safe form:

P_attention_linear = P_Q + P_KV + P_O

For the published DeepSeek-V2 dimensions:

query down:       5120 * 1536                         =   7,864,320
query up:         1536 * 128 * (128 + 64)             =  37,748,736
joint K/V down:   5120 * (512 + 64)                   =   2,949,120
K/V up:           512 * 128 * (128 + 128)             =  16,777,216
output:           5120 * 128 * 128                    =  83,886,080
attention linear parameters, one layer                = 149,225,472

Under M4.1’s convention that one multiply-add is two FLOPs, projecting B*T_q rows through those five matrix groups costs:

$$F_{proj}=2BT_q(P_Q+P_{KV}+P_O).$$

For B=2, T_q=1, the running fixture therefore performs:

2 * 2 * 1 * 149,225,472 = 596,901,888 projection FLOPs

That is a logical dense-matrix count. It excludes RMSNorm, RoPE, attention QK/PV work, softmax, communication, quantization, and backend padding. It also does not claim that every implementation materializes each matrix product as a separate kernel.

That ledger has three uses. It reconciles a checkpoint inventory, identifies which dimensions are sharded, and prevents the cache claim from being misstated as a parameter-count claim.

Persistent inference payload and learned parameters are different axes:

learned MLA factors:
    all five matrix groups remain resident somewhere

logical cache per token per layer:
    d_c + d_R scalars

temporary current-query state:
    H_q * (d_N + d_R) query features

optional derived state:
    absorbed key/output products, backend-packed weights, scheduler workspace

For one new BF16 row in one layer:

latent cache append:       (512 + 64) * 2 = 1,152 bytes
expanded naive append:     [128*(128+64) + 128*128] * 2 = 81,920 bytes
current query activation:  128*(128+64) * 2 = 49,152 bytes before liveness reuse

If the latent cache is replicated across R ranks, physical payload becomes R times the logical payload before pages, scales, metadata, and workspace. If it is uniquely owned, that multiplier disappears but remote demand and collectives enter the execution ledger. Loop 10 makes that placement trade explicit rather than hiding it inside d_c.

Training adds yet another ledger: gradients for trainable factors, optimizer moments, saved activations, and recomputation. Weight absorption is an inference reassociation, not a license to delete canonical trainable factors or their gradients.

Joint compression, alternatives, and information geometry

Let one uncompressed content map produce all per-head NoPE keys and values:

$$u_t=[k_{t,1}^{N};\ldots;k_{t,H_q}^{N};v_{t,1};\ldots;v_{t,H_q}] =W_{KV}^{full}h_t.$$

MLA constrains that map through a shared bottleneck:

$$u_t=W^{UKV}c_t^{KV},\qquad c_t^{KV}=W^{DKV}h_t,$$

so the effective map satisfies:

$$W_{KV}^{full}\approx W^{UKV}W^{DKV},\qquad \operatorname{rank}(W_{KV}^{full})\le d_c.$$

For native MLA, the factorization is learned end to end. It is not an SVD performed after training, and d_c is an architectural information bottleneck rather than a lossless encoding guarantee.

Alternative design choices expose different inductive biases:

The cache reduction is therefore bought with a learned rank constraint. If d_c is too small, the architecture cannot preserve all independent K/V directions; if it is too large, the runtime loses memory and bandwidth gains. The useful quantity is not rank alone but the end-to-end frontier across quality, cache bytes, decode traffic, and kernel efficiency.

Executable compression oracle

An oracle should test both factorization and state publication:

def joint_latent_reference(h, W_dkv, W_uk, W_uv, rmsnorm):
    raw = h @ W_dkv.T
    c_raw, k_rope_raw = split(raw, [d_c, d_R], axis=-1)
    c = rmsnorm(c_raw)
    k_nope = einsum("btd,hnd->bthn", c, W_uk)
    value = einsum("btd,hvd->bthv", c, W_uv)
    return c, k_rope_raw, k_nope, value

c, k_pe_raw, k_nope, value = joint_latent_reference(...)
assert c.shape == (B, T, d_c)
assert k_nope.shape == (B, T, H_q, d_N)
assert value.shape == (B, T, H_q, d_V)

# The cache contract publishes c and rotated k_pe, not k_nope/value.
cache.append(c, rope(k_pe_raw, position_ids))

For conversion, add reconstruction diagnostics:

relative Frobenius error of reconstructed K/V maps
singular-value energy retained at rank d_c
layer-output error before adaptation
post-adaptation short- and long-context quality

Low matrix error is evidence about initialization, not proof of equivalent attention after RoPE, softmax, cache quantization, and autoregressive reuse.

0x03 Why RoPE Must Be Decoupled

Without a position-dependent transform, a key expansion can be reassociated:

q @ (W_UK @ c_j)
    = (W_UK^T @ q) @ c_j

RoPE inserts a position-specific rotation:

query at t: R_t q
key at j:   R_j k

The key-side transform is no longer one static matrix shared by every cached position. Folding all R_j values into one position-independent query matrix would be invalid.

MLA therefore separates:

NoPE content dimensions:
    static linear maps, eligible for absorption

RoPE dimensions:
    position-dependent query/key rotations, scored explicitly

The cache and runtime must agree on:

d_c and d_R
RoPE pairing/layout
rope_theta and scaling configuration
position_ids/cache_position
whether k_rope is interleaved, split, quantized, or replicated

A layout mismatch is particularly dangerous: d_c + d_R can remain correct while the 64 rotary scalars are interpreted under a different pairing order. The result is valid memory access and wrong positional scores.

RoPE noncommutativity counterexample

The obstruction is stronger than “RoPE is nonlinear.” RoPE is linear for a fixed position, but the matrix differs with position. For one head, write the rotary score as:

$$(R_t q)^{\mathsf T}(R_j W^{KR}h_j) =q^{\mathsf T}R_t^{\mathsf T}R_jW^{KR}h_j.$$

The relative-position matrix R_t^T R_j depends on j-t. A static absorbed query matrix would need to satisfy:

$$W_{abs}=R_t^{\mathsf T}R_jW^{KR}\quad\text{for every cached }j,$$

which is impossible in general because the right side changes with j.

A two-dimensional counterexample makes the failure concrete. Let W^{KR}=I, query position t=0, and cached positions j=0 and j=1 with rotation angle theta != 0:

for j=0: R_0^T R_0 W_KR = I
for j=1: R_0^T R_1 W_KR = R(theta)

one static W_abs cannot equal both I and R(theta)

By contrast, the NoPE content score contains no position-dependent matrix:

$$q_N^{\mathsf T}W^{UK}c_j =(W^{UK\mathsf T}q_N)^{\mathsf T}c_j,$$

so it can be absorbed once for the current query and reused across all j.

This derivation also explains the design alternative. Applying RoPE to every content dimension would preserve a conventional attention interface but block full key-side absorption. Decoupling a smaller d_R positional subspace keeps relative-position information explicit while leaving the larger latent-content path eligible for reassociation. Conversion methods that use partial RoPE must therefore decide which dimensions retain positional rotation; that choice is a quality and adaptation decision, not a free systems rewrite.

0x04 Key-Side And Value-Side Weight Absorption

Key-side derivation

For query head i:

W_UK_i: [d_N, d_c]
c_j:    [d_c]
q_i:    [d_N]

k_i,j = W_UK_i @ c_j

Then:

$$q_i^{\mathsf T}k_{i,j} =q_i^{\mathsf T}W_i^{UK}c_j =(W_i^{UK\mathsf T}q_i)^{\mathsf T}c_j.$$

The naive parenthesization expands every historical latent row:

for each j:
    k_i,j = W_UK_i @ c_j
    score_i,j = dot(q_i, k_i,j)

The absorbed parenthesization transforms the one current query:

q_abs_i = W_UK_i^T @ q_i        [d_c]

for each j:
    score_content_i,j = dot(q_abs_i, c_j)

The historical read width changes from reconstructed H_q * d_N content to the shared d_c latent row.

Value-side derivation

For head i:

v_i,j = W_UV_i @ c_j
a_i,j = attention probability for key j

Linearity gives:

$$z_i =\sum_j a_{i,j}v_{i,j} =\sum_j a_{i,j}W_i^{UV}c_j =W_i^{UV}\left(\sum_j a_{i,j}c_j\right).$$

So the kernel can accumulate in latent space:

latent_out_i = sum_j a_i,j * c_j    [d_c]
head_out_i   = W_UV_i @ latent_out_i[d_V]

Let W_O_i be the output projection slice for head i:

$$y=\sum_i W_i^OW_i^{UV}\,latent\_out_i.$$

The product W_O_i W_UV_i can be precomputed or fused. Expansion occurs after history has collapsed to one weighted latent result per query head, not once per historical token.

Move elision

The two score terms share [B, H_q, T_q, T_k] but not feature width:

score_content  = q_abs  @ latent_cache.T
score_position = q_rope @ rope_cache.T
scores         = (score_content + score_position) * scale

A fused kernel can add the terms in the score tile. It does not need to:

broadcast k_rope over H_q
concatenate it with reconstructed content keys
write an MHA-shaped temporary

Absorption changes multiplication order. Move elision removes layouts that were introduced only to imitate an ordinary MHA API.

Batched absorption trace

The scalar derivation must survive batching, head axes, and the exact softmax axis. Use the following canonical shapes:

q_nope:       [B, H_q, T_q, d_N]
q_rope:       [B, H_q, T_q, d_R]
W_UK:         [H_q, d_N, d_c]
W_UV:         [H_q, d_V, d_c]
latent_cache: [B, T_k, d_c]
rope_cache:   [B, T_k, d_R]
W_O:          [H_q, d, d_V]

An explicit absorbed reference is:

# Move key expansion from T_k historical rows to T_q current rows.
q_abs = einsum("bhqn,hnc->bhqc", q_nope, W_UK)

score_content = einsum("bhqc,bkc->bhqk", q_abs, latent_cache)
score_rope = einsum("bhqr,bkr->bhqk", q_rope, rope_cache)
scores = (score_content + score_rope) * softmax_scale
probs = softmax(scores, dim=-1)  # normalize over T_k only

# Accumulate shared content separately for each query head.
latent_out = einsum("bhqk,bkc->bhqc", probs, latent_cache)
head_out = einsum("bhqc,hvc->bhqv", latent_out, W_UV)
y = einsum("bhqv,hdv->bqd", head_out, W_O)

The value and output factors can be contracted first:

W_VO = einsum("hdv,hvc->hdc", W_O, W_UV)  # [H_q,d,d_c]
y = einsum("bhqc,hdc->bqd", latent_out, W_VO)

These two implementations must agree before introducing quantization or a custom kernel. They expose three common bugs that scalar notation hides:

1. softmax over heads or flattened routes instead of T_k;
2. accidentally sharing latent_out across query heads even though the probability rows differ by head;
3. transposing the d_c axis differently in the key and value factors.

Training and backward contract

The reassociation is exact for linear inference contractions, but training has additional ownership:

canonical trainable factors:
    W_DKV, W_UK, W_UV, W_DQ, W_UQ, W_QR, W_KR, W_O

forward choices:
    materialized reference graph or algebraically absorbed graph

backward obligation:
    gradients of an absorbed product must reach every source factor

If W_VO = W_O W_UV is materialized outside autograd, training silently loses the factor gradients. A fused training kernel must either preserve the product inside the differentiable graph or implement the chain rule:

$$dW_O=dW_{VO}W_{UV}^{\mathsf T},\qquad dW_{UV}=W_O^{\mathsf T}dW_{VO}.$$

Native MLA training normally retains the explicit architecture factors. Serving may materialize derived products at load time because weights are frozen, but adapter merge/unmerge or hot reload must invalidate those products.

Decode crossover: when reassociation saves work

For the NoPE score term, compare the dominant multiply-add counts. The materialized order expands all historical keys and then scores them:

$$C_{mat}\approx 2BT_kH_qd_cd_N+2BT_qT_kH_qd_N.$$

The absorbed order transforms current queries and scores latent rows:

$$C_{abs}\approx 2BT_qH_qd_Nd_c+2BT_qT_kH_qd_c.$$

For one-token decode (T_q=1), equating the two gives a teaching crossover:

$$T_{k,\mathrm{cross}} \approx\frac{d_Nd_c}{d_cd_N+d_N-d_c},$$

when the denominator is positive and all other costs are ignored. With d_N=128, d_c=512, the historical expansion term dominates quickly, but the absorbed score also uses a wider 512-dimensional latent dot product than the 128-dimensional materialized NoPE score. This is why the systems win is not “fewer FLOPs in every phase.” It is a trade:

avoid:
    reconstructing and reading H_q * d_N historical key features
    reconstructing and reading H_q * d_V historical value features

accept:
    wider latent score/accumulation arithmetic
    current-query absorbed projections
    derived-weight and specialized-kernel requirements

Prefill has large T_q and different reuse, launch, and GEMM efficiency. A runtime may legitimately materialize head-shaped K/V for a compute-efficient prefill attention call while still publishing latent cache rows for decode. It must not publish those expanded temporaries as the persistent cache.

0x05 Reference Execution: Naive And Absorbed Paths

DeepSeek’s official V3 reference inference implementation exposes two paths. The exact symbol names can evolve, but the object boundary is clear.

Naive path

The naive path reconstructs per-head K/V and stores expanded caches:

def mla_naive_step(h_new, cache):
    q_nope, q_rope = project_query(h_new)
    c_kv, k_rope = project_joint_kv(h_new)

    k_nope, value = expand_latent_kv(c_kv)
    # k_nope: [B, T_q, H_q, d_N]
    # value:  [B, T_q, H_q, d_V]

    cache.k_nope.append(k_nope)
    cache.k_rope.append(k_rope)
    cache.value.append(value)

    scores = dot(q_nope, cache.k_nope) + dot(q_rope, cache.k_rope)
    probs = softmax(scores, dim=-1)
    return output_projection(probs @ cache.value)

This is useful as a readable reference and can reuse conventional attention operators. It does not realize the latent historical-cache payload.

Absorbed path

The absorbed path caches normalized latent state and positional keys:

def mla_absorbed_step(h_new, cache, weights):
    q_nope, q_rope = project_query(h_new)
    c_kv, k_rope = project_joint_kv(h_new)

    cache.latent.append(c_kv)      # [B, T_k, d_c]
    cache.rope.append(k_rope)      # [B, T_k, d_R]

    q_abs = absorb_key_projection(q_nope, weights.w_up_k)

    score_content = q_abs @ cache.latent.transpose(-2, -1)
    score_position = q_rope @ cache.rope.transpose(-2, -1)
    probs = softmax((score_content + score_position) * scale, dim=-1)

    latent_out = probs @ cache.latent
    return absorbed_value_output(latent_out, weights.w_up_v, weights.w_o)

The official model code makes this distinction concrete: the naive branch owns per-head k_cache and v_cache; the absorbed branch owns kv_cache with kv_lora_rank width plus a separate positional cache. It transforms the query with the key up-projection factors, scores latent and positional terms, performs softmax in float32 in the reference, accumulates latent values, and applies the value-side factors afterward.

Prefill publication contract

An engine may use a throughput-oriented prefill schedule and an absorbed decode schedule. That is valid only if prefill publishes exactly the state decode expects:

normalized c_kv rows
correctly rotated k_rope rows
same token positions and page order
same dtype/quantization layout
committed valid range and schema version

Storing expanded K/V during prefill does not automatically create a latent decode cache. A conversion would need to recover the exact latent state, which is generally not a free inverse operation.

Exact phase algorithms and commit state

Treat cache publication as a transaction. Projection may produce candidate rows before the scheduler decides which token positions become durable.

def prefill_mla(hidden, positions, pages, model, backend):
    q_nope, q_rope = model.project_query(hidden, positions)
    c_kv, k_rope = model.project_cache_rows(hidden, positions)

    # Attention may use a materialized or absorbed compute schedule.
    out = backend.prefill(q_nope, q_rope, c_kv, k_rope, causal=True)

    # Publication schema is independent of the temporary compute schedule.
    pages.write_uncommitted(c_kv, k_rope, positions)
    pages.commit(positions)
    return out


def decode_mla(hidden_one, position, page_table, model, backend):
    q_nope, q_rope = model.project_query(hidden_one, [position])
    c_new, kpe_new = model.project_cache_rows(hidden_one, [position])

    # Include the new row exactly once, under the backend's append convention.
    out = backend.decode(
        q_nope, q_rope, c_new, kpe_new,
        page_table=page_table,
        visible_length=position + 1,
    )
    page_table.commit(position)
    return out

For speculative verification with T_q > 1, candidate rows remain provisional:

verify N candidate input rows
compute target logits and candidate MLA rows
commit only the accepted input-row prefix
discard rejected suffix rows

if a correction output token differs from the rejected candidate:
    do not relabel the rejected row as the correction token
    compute the correction token's cache row on a valid later append path

Output-token commitment and cache-row commitment are related but not identical. The cache stores the hidden-state projection of the token actually processed at that position.

Paged and ragged address translation

Logical token index j becomes a physical cache address through the allocator:

$$page(j)=\left\lfloor\frac{j}{P}\right\rfloor,\qquad offset(j)=j\bmod P,$$ $$physical(j)=block\_table[request,page(j)]\cdot P+offset(j).$$

With a packed latent row, the byte address is:

$$addr(j)=base+physical(j)\cdot row\_stride.$$

The logical payload remains:

$$M_{logical}=BTL(d_c+d_R)b,$$

but the physical allocation is page-rounded and may include per-row scales.

def resolve_mla_row(block_table, request, token, page_size, row_stride):
    logical_page, offset = divmod(token, page_size)
    physical_page = block_table[request, logical_page]
    assert physical_page >= 0
    return (physical_page * page_size + offset) * row_stride

Ragged batches require one visible length per request. A flattened query row must carry its request id, query position, page-table base, and committed length. Reading to a batch-wide maximum without masking can expose padding or a different request’s page. Sparse APIs may encode each selected token directly as physical_page * page_size + offset; that is physical execution metadata, not a logical token id.

The cache publication record should contain at least:

request/cache identity
layer
logical token range
physical page and offset range
schema version and row stride
latent/RoPE dtype and scales
RoPE configuration hash
committed length or provisional transaction id

0x06 Numeric Cache Ledger

Use the preceding DeepSeek-V2 dimensions.

Naive expanded history

Per token per layer:

K scalars:
    H_q * (d_N + d_R)
    = 128 * (128 + 64)
    = 24,576

V scalars:
    H_q * d_V
    = 128 * 128
    = 16,384

total:
    40,960 scalars
    81,920 bytes at bf16
    80 KiB

Across 60 layers:

80 KiB * 60 = 4.6875 MiB per cached token

At T=4096 for one request:

4.6875 MiB * 4096 = 18.75 GiB

Absorbed latent history

Per token per layer:

d_c + d_R = 512 + 64 = 576 scalars
576 * 2 bytes = 1152 bytes = 1.125 KiB

Across 60 layers:

1.125 KiB * 60 = 67.5 KiB per cached token

At T=4096:

67.5 KiB * 4096 = 270 MiB

The internal naive-to-absorbed ratio for these dimensions is:

$$\frac{40960}{576}\approx71.1.$$

This ratio is not DeepSeek-V2’s paper denominator. The paper’s reported 93.3% reduction compares its architecture with a named baseline model and should be cited as that source-specific result. The ledger above compares two execution representations under one stated set of dimensions.

Physical-residency correction

For deployment, measure:

$$M_{physical} =M_{payload}+M_{scales}+M_{padding}+M_{metadata}+M_{replicas}+M_{workspace}.$$

An allocator can reserve fixed pages, a quantized format can add scale words, and data/tensor parallelism can replicate latent or positional rows. Those terms do not invalidate the architecture formula; they explain why profiler memory will not equal it exactly.

0x07 Phase-Specific Execution Modes

One MLA equation can have different high-performance schedules.

Prefill

Prefill has many query rows. Large matrix operations can justify reconstructing per-head features inside tiles or using an MHA-like mode while still publishing latent persistent state.

prefill objective:
    maximize throughput over many query rows

persistent result:
    write latent + RoPE cache rows for future decode

Decode

Ordinary decode has T_q = 1. Historical-cache reads dominate, so the absorbed representation should remain compressed:

decode objective:
    transform current Q once
    stream latent + RoPE history
    avoid full historical K/V reconstruction

MTP/speculative query chunks

FlashMLA’s API uses s_q for query tokens per sequence. Ordinary decode uses s_q = 1; multi-token prediction or speculative verification can use s_q > 1. That changes work per cache row and can move the kernel across a roofline boundary. It does not change cache semantics.

”MHA mode” and “MQA mode” in FlashMLA

Current FlashMLA documentation uses these terms for execution layouts:

MLA MQA mode:
    key dimension 576
    value dimension 512
    one shared latent/RoPE cache stream used by many query heads

MLA MHA mode:
    per-head key dimension 192 or 128
    value dimension 128

These labels describe how the MLA computation is presented to a kernel. They do not mean the checkpoint changed back and forth between ordinary MQA and MHA. As of the source version checked for this article, FlashMLA’s support matrix lists dense decoding on SM90 in MQA mode with BF16 cache, sparse decoding on SM90/SM100 in MQA mode with FP8 cache, dense prefill on SM100 in MHA mode, and sparse prefill on SM90/SM100 in MQA mode. This is a current implementation matrix, not a permanent property of MLA.

0x08 FlashMLA Runtime ABI

An optimized kernel requires more than Q and a cache pointer. The official FlashMLA decode example has a preparation call and a per-layer call.

Pinned Source Boundary For This ABI

The runtime contract below was re-audited on 2026-08-03 against:

repository: deepseek-ai/FlashMLA
commit:     15f13e5030374295491c5ce31b02d7e63a7772c6
interface:  flash_mla/flash_mla_interface.py

At this revision, flash_mla_with_kvcache accepts a query, a paged cache, dense block-table/length metadata or sparse physical indices, value width, scheduler metadata, cache-format flags, optional extra sparse segments, and attention sinks. It mutates the scheduler metadata object with returned metadata/split state. Dense and sparse modes have different preconditions:

dense:
    block_table and cache_seqlens identify paged visible history
    sparse-only indices/extra segments must be absent

sparse:
    physical indices identify selected rows
    causal=False in this ABI
    FP8 cache is required by the checked sparse decode path
    optional topk_length bounds the valid prefix of each index row

The interface checks that scheduler metadata was built for the same page size, cache dtype, sparsity/top-k, extra-cache shape, and head configuration. That is the executable reason metadata cannot be cached under “same batch size” alone.

The evidence bundle records the commit, path, downloaded-file SHA-256, and related framework pins. A measured result must record the exact revision actually built and the backend logged at runtime.

Scheduler metadata before the layer loop

At the pinned revision, metadata creation is lazy. get_mla_metadata accepts legacy positional arguments only for source compatibility, ignores them, and returns an empty FlashMLASchedMeta plus a None placeholder:

sched_meta, num_splits_placeholder = get_mla_metadata()
assert num_splits_placeholder is None

The first flash_mla_with_kvcache call derives configuration from the actual Q and cache tensors and stores returned tile metadata and Split-K state inside that object. Reuse is legal only while batch size, s_q, query heads, cache page size, K heads, causal mode, cache format, sparse top-k, extra-cache page size, and extra top-k remain compatible. The pinned interface additionally documents that cache_seqlens, topk_length, and extra_topk_length values must remain the same when one scheduler object is reused.

# First layer/call initializes sched_meta; later compatible calls update and
# reuse its internal tile_scheduler_metadata and num_splits fields.
out, lse = flash_mla_with_kvcache(
    q, k_cache, block_table, cache_seqlens, head_dim_v=512,
    tile_scheduler_metadata=sched_meta,
    num_splits=None,
    softmax_scale=scale,
    causal=False,
    is_fp8_kvcache=False,
)
assert sched_meta.have_initialized

Older FlashMLA revisions eagerly computed metadata in get_mla_metadata from sequence lengths and head counts. That historical signature must not be copied into code claiming the 2026-08-03 pin. The source trace records both the positive current contract and this superseded boundary.

That reuse has a strict scope. Metadata may be shared across layers only when the layers share the relevant cache layout and active request geometry. CUDA graph capture commonly reserves persistent metadata buffers and copies the active prefix into them; buffer capacity, active view length, and captured addresses must all remain valid during replay.

Layer call

Current dense call, omitting only defaulted optional arguments:

out, lse = flash_mla_with_kvcache(
    q,
    k_cache,
    block_table,
    cache_seqlens,
    head_dim_v=512,
    tile_scheduler_metadata=sched_meta,
    num_splits=None,
    softmax_scale=scale,
    causal=False,
    is_fp8_kvcache=False,
    indices=None,
)

The contract contains:

Returning lse is not incidental. Split-K partial outputs cannot be averaged. Their maxima, denominators, and numerators must be merged with the exact online softmax algebra derived in M4.5. The ABI exposes enough normalization state for that execution family.

Metadata validity

Cached scheduler metadata is invalid if the active batch shape, sequence lengths, sparse top-k, cache format, or head configuration changes. Reusing stale metadata can assign too little work, read invalid pages, or merge the wrong number of partial states.

An API flag is not an allocation receipt. Prove the active path with:

backend-selection log
loaded extension/library revision
kernel symbol or profiler trace
actual cache stride and dtype
metadata shape and num_splits
fallback counter and reason

0x09 Why MLA Decode Can Become Compute-Bound

“Decode is memory-bound” is a useful first approximation, not a law. The official FlashMLA deep dive derives the MLA attention intensity for:

h_q: query heads
s_q: query tokens per request
s_k: cached tokens, with s_k much larger than h_q * s_q
d_k: key width
d_v: value width

QK plus PV work is approximately:

$$F\approx2h_qs_qs_k(d_k+d_v).$$

For a shared MQA-mode cache in bf16, dominant cache traffic is approximated as:

$$D\approx2s_kd_k\quad\text{bytes}.$$

Therefore:

$$I=\frac{F}{D} \approx h_qs_q\frac{d_k+d_v}{d_k} \approx2h_qs_q\quad\text{FLOPs/byte}.$$

The important difference from ordinary one-head-per-query decode is that many query heads reuse one wide latent cache row. With h_q=128 and s_q=1, the source’s rounded approximation is about 256 FLOPs/byte.

The same deep dive uses an H800 SXM5 with 3.35 TB/s peak bandwidth and about 865 TFLOP/s practical throttled compute, giving a ridge near 258 FLOPs/byte. It classifies its 128-head deployment near/on the compute-bound side and designs the kernel accordingly. This is deliberately source-qualified:

change h_q, s_q, dimensions, clocking, GPU, cache dtype, or reuse efficiency
and the bottleneck can move

The exact documented MQA-mode dimensions d_k=576, d_v=512 give a ratio (d_k+d_v)/d_k of about 1.89 rather than exactly 2. The result sits close enough to the stated ridge that profiler counters matter more than the slogan.

0x0A Register Pressure And Seesaw Scheduling

FlashAttention-3’s ping-pong schedule overlaps two work pipelines, but the FlashMLA deep dive identifies a register constraint for its output accumulator.

output tile:
    64 * 512 = 32,768 accumulator elements

32-bit register storage:
    32,768 * 4 bytes = 128 KiB

H800 registers per SM in the source analysis:
    65,536 32-bit registers = 256 KiB

One full output matrix consumes half the SM register file before accounting for other live state. The source concludes that two full output accumulators cannot support the ordinary ping-pong design.

The new kernel splits output vertically:

O_L: [64, 256] owned by warpgroup 0
O_R: [64, 256] owned by warpgroup 1

It also splits value blocks and interleaves two K/V blocks. The warpgroups alternate QK, max/exp rescaling, and PV work while sharing the running online softmax maximum. DeepSeek calls this seesaw scheduling.

Source: DeepSeek FlashMLA, “A Deep-Dive Into the New Flash MLA Kernel,” pinned snapshot. The schedule is mathematically equivalent to online-softmax accumulation; the split is a resource/work-pipeline decision.

The schedule preserves exact attention because each update rescales prior partial state when the running maximum changes. It is not legal to interleave blocks by independently normalizing them and adding final outputs.

Additional source-described pipeline decisions include:

• split a 64 x 576 K block into nine 64 x 64 TMA transfers and begin GEMM as soon as each piece arrives;
• use an evict-first cache hint for streamed TMA data in that implementation;
• overlap splitkv_mla and the combine kernel with programmatic dependent launch;
• use a tile scheduler to balance variable request/block jobs across SMs.

The deep dive reports up to 80% of its throttled Tensor Core peak and about 3 TB/s in its H800 experiments. The README at the pinned FlashMLA revision reports up to 660 TFLOP/s in compute-bound dense decode and 3000 GB/s in memory-bound configurations on H800 SXM5 with CUDA 12.8. These are source-reported results from their respective benchmark configurations, not measurements reproduced here or claims about today’s default backend.

0x0B FP8 Cache Format And Sparse Selection

Exact 656-byte FP8 row

Current FlashMLA documentation defines its “FP8 with scale” cache row as:

The row total follows directly from those three stored segments:

$$M_{\mathrm{FP8,row}}=512(1)+4(4)+64(2)=656\ \text{bytes}.$$

A fully bf16 latent plus RoPE row is:

(512 + 64) * 2 = 1152 bytes

The documented FP8 row ratio is:

$$\frac{656}{1152}\approx0.569,$$

or about 43.1% fewer bytes than the 1152-byte bf16 row. It is not 50% smaller because scales remain fp32 and RoPE remains bf16.

The current sparse decode kernel dequantizes the latent cache to bf16, performs the attention computation in bf16, and returns bf16 output. This separates:

storage format:
    E4M3 latent + fp32 scales + bf16 RoPE

compute format:
    bf16 attention

M8.6 owns the broader quantization theory. M5.8 owns the fact that scale words, unquantized positional state, page stride, and dequantization belong to the MLA cache ABI.

Sparse indices are physical execution metadata

For sparse decode, current FlashMLA documents:

indices: [B, S_q, topk]

Each entry encodes a physical page-block index plus an offset within that block. At the pinned decode interface, topk_length[B] can bound the valid left prefix of each query’s index row. Do not generalize -1 sentinel behavior across entry points: the pinned sparse-prefill doc explicitly permits -1 or values outside s_kv, while the decode contract instead documents physical indices and valid-prefix lengths. Because each valid selected location is already encoded, sparse decode does not need the same logical block-table lookup for that row.

Sparse attention changes which historical tokens participate. It is not a numerically approximate implementation of dense attention under the same mask. The validation ladder must distinguish:

Current repository results report sparse FP8 decode performance on H800 and B200 and sparse prefill results on H800/B200. Those numbers must stay attached to the repository revision, GPU, CUDA version, shapes, top-k, and cache format.

Dense MLA, sparse MLA, and DSA are different contracts

Dense MLA changes the representation of every visible historical row. Sparse MLA additionally changes the visible set. A learned indexer such as DeepSeek Sparse Attention may produce top-k candidates, but selection policy is not an algebraic consequence of weight absorption.

dense MLA:
    visible set = all tokens allowed by mask/retention policy

sparse MLA:
    visible set = selected physical rows plus any mandatory local/sink segment

FP8 sparse MLA:
    visible set changes and storage representation changes

The evaluation must therefore factor error sources:

$$\Delta_{total} =\Delta_{absorption} +\Delta_{numeric} +\Delta_{selection} +\Delta_{runtime},$$

as a diagnostic decomposition rather than an assumption of additive model error. Hold the selected set fixed to test quantization; use dense visibility to test absorption; evaluate recall and downstream quality to test selection.

M5.4 owns which tokens may be discarded or selected. M5.8 owns how selected physical rows are represented and consumed by an MLA backend. This boundary prevents a kernel speedup from being reported as if it preserved dense attention semantics automatically.

0x0C Derived Weights, Parallelism, And Ownership

Materialize or contract absorbed weights

Absorption introduces products such as:

key side:
    W_UK_i^T applied to q_nope_i

value/output side:
    W_O_i @ W_UV_i

An engine can pre-materialize derived matrices at model load or contract source factors at runtime.

A derived-weight cache key needs:

base checkpoint/version
active adapter or merged-delta state
dtype and quantization packing
tensor-parallel shard layout
source factor versions
kernel/backend layout

LoRA/adapters, weight reload, quantization repacking, or resharding can make a shape-correct absorbed matrix stale.

MLA parallelism is not ordinary head sharding

The latent cache is shared across reconstructed query heads. Useful execution axes include:

For one request, define the logical latent payload:

$$M_{logical}=TL(d_c+d_R)b_c.$$

If each of R tensor-parallel ranks retains a complete copy:

$$M_{physical,replicated}=R\,M_{logical}+M_{pages}+M_{metadata}+M_{workspace}.$$

If history is partitioned without replicas, core cache residency returns to M_logical, but every query must reach all history partitions and merge exact softmax state. Neither policy is “free MLA memory.”

Derived products need their own ledger. A literal per-head value/output product has H_q*d*d_c scalars. For the running dimensions in BF16:

H_q * d * d_c * 2
    = 128 * 5120 * 512 * 2
    = 671,088,640 bytes
    = 640 MiB per layer

That large result explains why “precompute the absorbed matrix” is a design choice, not an automatic optimization. A runtime may retain factorized source weights, contract smaller pieces, shard the derived product, or fuse the final contractions. The artifact manifest must report which representation exists on each rank.

For sequence-split attention, rank r produces local online-softmax state:

m_r: local row maximum
l_r: local denominator under m_r
o_r: local latent numerator under m_r

The exact merge is:

$$m=\max_r m_r,$$ $$l=\sum_r e^{m_r-m}l_r, \qquad o=\sum_r e^{m_r-m}o_r, \qquad O=o/l.$$

The absorbed value/output transform follows the normalized global latent result. Averaging rank-local outputs is wrong.

DeepSeek’s published serving description uses data-parallel attention for its decode configuration rather than tensor-parallel attention, which keeps all 128 query heads and one request’s latent cache together for the FlashMLA intensity argument. Other deployments may choose differently. The stable lesson is to state cache ownership and collective placement before quoting kernel numbers.

Distributed ownership and communication slots

For request/data-parallel attention, rank r owns complete attention state for a subset of requests:

request dispatch -> rank-local latent pages -> all local query heads

That avoids an attention collective inside each layer but requires request and KV placement to remain aligned. If MoE layers use a different parallel axis, the block can still incur dispatch or all-to-all communication outside the attention kernel.

Tensor-parallel projection ownership can split output rows or input columns:

column-parallel query/KV up-projections:
    produce local query-head features

row-parallel output projection:
    consumes local head output and reduces hidden contributions

latent cache:
    replicate, shard by request, or shard by sequence explicitly

Replicating the latent cache can be reasonable because it is compact, but the physical ledger must multiply by replica count. Sharding d_c is more subtle: each content score becomes a reduction across latent-dimension shards, while sequence sharding requires the exact max/denominator/numerator merge above.

Communication belongs in named slots:

Pinned runtime source trace

The article’s implementation claims were checked at fixed revisions rather than against moving main pages:

The full 40-character commits and file hashes live in the M5.8 source manifest. These implementations agree on the architectural idea but not on one universal ABI. For example:

FlashMLA dense decode:
    combined paged cache row plus scheduler metadata

FlashInfer paged MLA:
    plan(qo_indptr, kv_indptr, kv_indices, dims, page_size)
    run(q_nope, q_pe, ckv_cache, kpe_cache)

framework integration:
    builds allocator and graph-capture metadata, selects backend,
    writes new cache rows, and handles prefill/verify fallbacks

The stable contract is tensor semantics plus cache identity, not parameter names. A backend oracle should execute a shape matrix and record dispatch:

for phase in ["prefill", "decode", "verify"]:
    for cache_dtype in ["bf16", "fp8"]:
        for history in [1, 127, 128, 129, 4096]:
            result = engine.run(case(phase, cache_dtype, history))
            assert result.backend == expected_or_documented_fallback(case)
            compare_to_absorbed_reference(result)
            record(result.kernel_symbol, result.cache_stride, result.metadata)

Support matrices evolve. A documentation table is evidence of intended support; successful dispatch and profiler evidence prove the path actually ran.

0x0D Native MLA, Conversion, And Variant Boundaries

Paper-by-contract comparison

Native and converted MLA may share an inference-shaped graph while making very different claims:

TransMLA’s useful systems proposition is that grouped K/V maps can be expressed through low-rank factors and served with an MLA-style cache after adaptation. That does not mean every GQA checkpoint is exactly low rank at the selected d_c, nor that converting weights alone activates a latent backend.

MHA2MLA must first address full RoPE. If all K dimensions are position-rotated, the key up-projection cannot be fully absorbed. It selects a smaller rotary subspace and treats the remainder as NoPE content. Candidate policies include retaining high-frequency pairs, low-frequency pairs, uniform samples, or pairs with large source-reported contribution scores.

After choosing NoPE key dimensions, a conversion can factor key and value maps separately or jointly. With a joint matrix:

$$W_{joint}=[W_{K,N};W_V]\approx U_r\Sigma_rV_r^{\mathsf T},$$

one balanced initialization is:

$$W^{UKV}=U_r\Sigma_r^{1/2},\qquad W^{DKV}=\Sigma_r^{1/2}V_r^{\mathsf T}.$$

Splitting the rows of W^{UKV} recovers initial key and value up-projections. A separate-SVD alternative allocates independent ranks to K and V, but stores or combines two latent directions. Joint SVD uses one rank budget and captures cross-structure, while forcing both tasks through the same subspace.

Conversion evaluation needs more than perplexity after fine-tuning:

before adaptation:
    matrix reconstruction, layer output, attention-logit, and cache-row checks

during adaptation:
    data manifest, token budget, optimizer schedule, frozen/trainable tensors,
    context-length curriculum, and loss recovery

after adaptation:
    short/long-context quality, retrieval/position tasks, generation stability,
    cache bytes, TTFT, TPOT, goodput, and executed backend

Paper-reported quality or speed applies to the paper’s checkpoint, adaptation budget, hardware, software, and workload. It is not a generic guarantee for another model converted at the same nominal rank.

The adaptation split is immutable evidence, not an informal dataset name:

conversion_manifest:
    source checkpoint digest and tokenizer digest
    method and paper-contract version
    partial-RoPE pair indices, if applicable
    joint/split factorization, rank, seed, and reconstruction receipt
    train/validation/test document IDs and content hashes
    deduplication and contamination policy
    sequence-length curriculum and packing policy
    optimizer, precision, scheduler, token budget, and checkpoints
    frozen/trainable parameter inventory

Changing the retained RoPE pairs, rank, split, or token budget creates a new conversion candidate. It is not a retry of the same experiment.

Low-rank weights alone do not prove low cache traffic. A conversion claim needs:

checkpoint transformation
adaptation data and budget
short- and long-context quality
cache schema actually stored
selected prefill and decode backends
physical bytes and traffic

This is the same separation used throughout Transformer System: mathematical representability, trained quality, and systems realization are different claims.

0x0E Evaluation And Parity Harness

Layer-level numerical ladder

Use fixed weights, hidden rows, positions, and masks:

1. naive expanded reference in float32 where feasible;
2. naive BF16 path;
3. absorbed BF16 reference;
4. optimized dense BF16 kernel;
5. quantized-cache path with the same visible token set;
6. sparse path evaluated separately under its selected set.

Compare:

content scores before softmax
position scores before softmax
combined scaled scores
row max and log-sum-exp
latent weighted output
post-absorption layer output
next-token logits over multiple decode steps

One-step output parity can miss cache corruption that compounds only after new rows are appended.

Cache-state assertions

After prefill and every decode append:

cache width == d_c + d_R or documented packed stride
latent row equals normalized reference c_kv
RoPE row uses the correct position and pairing layout
valid length/commit state advances atomically
block table or sparse indices resolve to the intended token
dtype and scales match cache schema

Performance matrix

Record at least:

Collect kernel time, HBM bytes, achieved FLOPs, occupancy/register pressure, workspace, end-to-end TPOT, TTFT when prefill is involved, and goodput under the same service objective. A peak TFLOP/s number does not prove a better serving configuration.

Release and rollback contract

An MLA release changes persistent request state. A process cannot switch cache schema midway through a request merely because a health monitor selects another backend.

The immutable release bundle contains:

model/checkpoint digest and architecture dimensions
derived absorbed-weight digest and source-factor versions
RoPE configuration and pairing layout
cache schema, row stride, page size, dtype, scales, and allocator version
backend build revision, GPU support matrix, and fallback policy
validation corpus/split manifest and tolerance policy
benchmark workload, hardware, software, clocks, and SLO

Use staged acceptance:

1. offline: naive versus absorbed float/BF16 parity over prefill and repeated append; packed-cache and page-address tests; sparse tests with fixed indices;
2. shadow: duplicate requests without serving candidate outputs, compare logits/cache hashes and collect backend/fallback counters;
3. canary: admit new requests only, bounded traffic, monitor correctness, OOM, TTFT, TPOT, goodput, and tail latency;
4. rollout: increase admission by cohort while old-schema requests drain;
5. closure: archive source pins, artifacts, measurements, and decision.

Rollback is request-cohort aware:

safe fast rollback:
    stop admitting new requests to candidate runtime
    route new requests to prior model/cache schema
    let existing candidate requests drain on their compatible runtime

state migration rollback:
    allowed only with a tested exact conversion between cache schemas

unsafe rollback:
    reinterpret latent pages as expanded K/V, or change RoPE/cache packing
    for an in-flight request

Release gates should fail closed on unknown backend dispatch, mismatched cache schema, stale derived weights, unsupported GPU/dtype combinations, or missing benchmark provenance. A slower documented fallback can be acceptable; an unobserved fallback invalidates the performance claim.

0x0F Failure Lab

0x10 Claims, Handoffs, And Publish Gate

Externally grounded claims:

• DeepSeek-V2 introduces MLA with joint low-rank K/V compression and decoupled RoPE and reports a large source-specific KV-cache reduction.
• DeepSeek’s reference V3 inference code exposes naive expanded and absorbed latent-cache execution paths.
• FlashMLA’s current official repository exposes dense and sparse prefill/decode kernels under a versioned GPU/cache-format support matrix.
• Its decode ABI includes scheduler metadata, paged cache state, sequence lengths, Split-K count, optional sparse physical indices, output, and LSE.
• Its official deep dive derives the MLA intensity, register limit, seesaw schedule, fine-grained TMA pipeline, and source-specific H800 results.

Local derivations and synthesis:

• latent BF16 payload is (d_c+d_R)b per token per layer;
• the stated DeepSeek-V2 dimensions give a 71.1x internal payload ratio between naive expanded history and absorbed latent history;
• the official 656-byte FP8 row is about 43.1% smaller than its 1152-byte BF16 latent-plus-RoPE counterpart;
• cache compression is realized only when prefill publication, decode kernels, page addressing, and output absorption preserve the latent representation;
• sparse MLA changes the visible token set and must not be evaluated as a simple numerical approximation of dense attention.

Handoffs:

M4.5/M4.6:
    exact Split-K/online-softmax merge and asynchronous GPU pipelines

M5.3:
    page tables, physical cache allocation, and exact prefix identity

M5.6:
    transfer/placement of the representation between prefill and decode owners

M7:
    s_q > 1 speculative or multi-token verification shapes

M8.6:
    FP8 numerical formats, scale selection, and quantization evaluation

X1.3 checkpoint:
    multimodal packing, media/prefix cache identity, batching, admission,
    cancellation, and recovery state

Readers following the multimodal track continue to X1.3. It applies M5’s cache ownership and scheduler invariants to media bindings, expanded row ranges, prefix K/V, and separately reusable cross-attention memory.

Publish gate

References

Local source articles:

• Transformer Series A28 - DeepSeek MLA
• M3.4 - Rotary Position Embedding
• M4.5 - Decode Attention Kernels and Backend Boundaries
• M4.6 - FlashAttention-3/4 Hardware Pipelines
• M5.3 - KV Memory Layout and Exact Reuse
• M5.6 - Prefill/Decode Disaggregation and KV Placement
• M5.7 - MQA and GQA
• M8.6 - Live-Data Quantization

Primary and official sources:

• DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
• DeepSeek-V3 Technical Report
• DeepSeek-V3 reference inference implementation, pinned
• FlashMLA README and support matrix, pinned
• FlashMLA pinned interface inspected on 2026-08-03
• A Deep-Dive Into the New Flash MLA Kernel, pinned
• DeepSeek open inference-system overview, pinned historical deployment description
• SGLang FlashMLA backend, pinned
• vLLM FlashMLA backend, pinned
• FlashInfer MLA core, pinned
• TransMLA: Multi-Head Latent Attention Is All You Need
• MHA2MLA: Enabling DeepSeek’s Multi-Head Latent Attention in Any Transformer-based LLM
• Hardware-Centric Analysis of Multi-Head Latent Attention