M6.5 - Multi-Adapter Serving: Kernels, Residency, Batching, and KV Compatibility

0x00 Overview: A Small Adapter Creates A Large Runtime Contract

M6.3 defined a low-rank update. M6.4 compared ways to train and parameterize it. Neither article answers the production question:

How can one resident base model serve requests for many adapters without corrupting request state or fragmenting every batch by adapter identity?

0x00A The Runtime Problem Predates LoRA

Parameter-efficient training and parameter-efficient serving are different problems. Reducing the task-specific artifact from a complete model copy to a small parameter set does not tell an inference engine how to register that state, batch requests that select different states, place those states in memory, or preserve them across an autoregressive request.

The systems lineage makes this separation clearer:

Stage	Task-specific object	Shared opportunity	New systems problem
full fine-tuning	one complete model per task	little after deployment	model copies, swaps, and fragmented capacity
early parameter-efficient transfer	prompts, bottlenecks, masks, biases, or deltas	one frozen backbone	heterogeneous task operators and artifact registration
PetS	registered PET type plus task parameters	task-agnostic base operators	coordinated task batching and operator dispatch
LoRA	two low-rank factors and scaling metadata per target	one large base projection	heterogeneous low-rank execution inside one batch
Punica and S-LoRA	many independently identified LoRA objects	one resident base model	segmented kernels, residency, paging, and adapter-aware scheduling
stateful generation	adapter identity plus request-local history	batched prefill and decode	adapter/KV compatibility, in-flight lifetime, and safe version changes

PetS is the important pre-LoRA systems ancestor here. It does not define LoRA and does not implement a modern decoder-only serving stack. It identifies the more general decomposition that later LoRA systems inherit: shared model operators, task-specific parameter objects, a registry/loader, a scheduler, and a specialized operator library. In other words, making task state small exposes a systems interface; it does not remove one.

Source: PetS, USENIX ATC 2022, Figure 4, reproduced from the official open-access paper. PetS serves several parameter-efficient methods and predates LoRA-specific multi-tenant systems. The figure establishes historical task-state decomposition; it is not evidence for Punica, S-LoRA, PEFT, or vLLM implementation details.

LoRA then gives the task-specific branch a particularly regular algebraic shape. Punica asks how different LoRA factors can share the base projection without a launch per adapter. S-LoRA asks how a much larger collection of adapter objects can coexist with dynamic KV state and tensor parallelism. Modern runtimes such as vLLM expose adapter identity through request, residency, kernel-metadata, and load/unload interfaces. These are related responses to different bottlenecks, not interchangeable names for one system.

0x00B Owner And Boundary Contract

M6.5 consumes six completed contracts instead of deriving them again:

Consumed owner	Contract used here	What M6.5 must not reteach
M6.3	vanilla LoRA factors, target modules, checkpoint identity, merge semantics	ordinary LoRA forward/backward and checkpoint construction
M6.4	method-specific representation and compatibility axes	the variant catalog and its training recipes
M6.7	rank, subspace, optimization, and retention diagnostics	low-rank theory or rank-selection claims
M5.1	request-local KV append, commit, and release lifecycle	the basic autoregressive cache state machine
M5.2	admission, iteration planning, batching, fairness, and preemption	the generic continuous-batching scheduler
M5.3	physical KV pages, cache identity, copy-on-write, and exact reuse	generic PagedAttention allocation and prefix reuse

This article owns the adapter-specific serving extension:

adapter registry identity and revision
request -> adapter selection
homogeneous and segmented adapter execution
adapter device/host residency and page lifetime
adapter-aware admission, pinning, eviction, and hotswap
adapter + KV + prefix compatibility
runtime telemetry, failure detection, and recovery

It defers learned routing, sharing, and continual composition to M6.9, and it defers general quantized packaging, deployment evaluation, release, and rollback to M8. A scheduler decision appears here only when adapter state changes its feasibility or correctness. A KV mechanism appears here only when adapter provenance changes cache identity or object lifetime.

0x00C Evidence And Claim-Status Contract

Status	Meaning in this article
Source-local	The hand-written A22 source or another completed Transformer System owner supplies terminology, context, or a consumed contract; it is not independent verification.
Externally verified	A primary paper or official proceedings source supports the scoped claim.
Official implementation	A revision-pinned project source demonstrates one concrete API, state layout, or execution path; it is not a universal runtime contract.
Rewrite interpretation	A Transformer System derivation, invariant, toy trace, or design recommendation assembled from the frozen sources.
Source-reported	A benchmark, scale, or latency result reported by its authors and not reproduced here.
Uncertain	A version-, hardware-, workload-, or method-dependent claim that remains explicitly unresolved.

Loop 1 freezes LoRA v2, the official PetS paper, Punica v1, the S-LoRA MLSys paper, and revision-pinned Punica, S-LoRA, PEFT, and vLLM source trees. Later loops may execute bounded local oracles against those contracts. This article does not claim to reproduce a source benchmark, validate a production deployment, or prove that one runtime policy is optimal.

During training, an adapter can look like a small pair of tensors saved beside a base-model identifier. Serving changes the unit of analysis. Requests arrive concurrently, select different adapters, contribute different numbers of prompt or decode rows, and retain KV state across iterations. The runtime must make adapter identity part of execution ownership.

The naive story is:

load one base model
load a small adapter
select adapter per request

The actual system must coordinate:

base-model residency
adapter metadata and compatibility
GPU/CPU adapter placement
heterogeneous low-rank kernels
continuous batching
request KV ownership
prefill/decode scheduling
hotswap and eviction safety
latency and memory SLOs

This article treats an adapter as live serving state, not as a file attached after the model has already been scheduled. Punica and S-LoRA are useful because they expose two different parts of that state: heterogeneous adapter kernels and a memory system for many adapter/KV objects.

M5.2 owns the ordinary iteration plan, queue policy, admission, fairness, and completed-service accounting. M6.5 consumes that plan and adds adapter identity, residency, and kernel feasibility; it does not redefine continuous batching.

0x01 Prerequisites: From One Adapter To Multi-Tenant Execution

0x01A The Single-Adapter Baseline

For one linear layer and one adapter:

$$Y=XW^\top+sXA^\top B^\top$$

there are two common deployment modes:

merged:
    materialize W' = W + sBA
    run one ordinary base-shaped projection

live:
    keep W shared
    execute the low-rank branch for each forward pass

Merged execution is simple when one adapter is fixed, but it creates a different effective base weight for each adapter. Keeping hundreds of merged copies would duplicate the base model. Live execution preserves one shared base, but adapter identity now affects every targeted layer.

0x01B Why Ordinary Batching Assumes Shared Weights

An ordinary dense batch works because every row uses the same matrix:

$$Y=XW^\top$$

If row groups use different adapters, only the base term remains homogeneous:

$$Y_j=X_jW^\top+s_{a_j}X_jA_{a_j}^\top B_{a_j}^\top$$

The batch therefore contains:

one shared large GEMM:
    every row uses W

many heterogeneous low-rank operations:
    row j uses the factors, rank, and scale for adapter a_j

Splitting the entire batch by adapter wastes the shared base-model opportunity. Running a Python loop over adapters preserves correctness but creates many tiny launches. Multi-adapter serving needs a kernel interface that consumes row-to-adapter metadata directly or groups rows while retaining an inverse permutation.

0x01C Four State Lifetimes Must Be Separated

Serving one separately fine-tuned model per task duplicates the base weights and fragments accelerator capacity. Live adapters introduce a hierarchy instead:

State class	Examples	Lifetime
shared model state	base weights, tokenizer/config, common kernels	worker or deployment
adapter state	factors, rank, scale, target modules, base revision	residency/cache entry
request state	adapter identity, sampling state, KV handle	request
scheduler state	active rows, adapter reservations, page ownership	iteration and queue

Correctness requires these lifetimes to agree. Evicting an adapter while a kernel still references its device slot is a use-after-free. Reusing a KV prefix under a different K/V-changing adapter is semantic cache corruption. Loading the right adapter file against the wrong base revision is a checkpoint compatibility failure.

0x01D Residency Is Different From Adapter Size

An adapter may be tiny relative to the base model and still miss a latency objective. A cold request can require:

registry lookup
    -> host or object-store read
    -> validation and tensor materialization
    -> host-to-device transfer
    -> device-slot publication
    -> first kernel use

The relevant quantities are not only bytes per adapter:

GPU-resident adapter capacity
host-resident cache capacity
cold-load latency and transfer bandwidth
number of distinct adapters per iteration
rank distribution and kernel workspace
in-flight references that prevent eviction

Adapter placement is therefore a cache and admission problem, not merely a file-loading API.

0x01E Prefill, Decode, And KV State Change The Batch Shape

Prefill can contribute many rows from one request, so an adapter may own a useful matrix-matrix sub-batch. Decode often contributes one row per active request, creating many tiny adapter segments:

prefill:
    request a -> [S_a,d] rows

decode:
    request a -> [1,d] row per iteration

Meanwhile the KV cache persists across iterations. Directly adapting K or V makes cache dependence obvious, but target-module names alone are not a proof of compatibility. An adapter applied to an earlier attention output or MLP can change the residual stream that later layers project into K/V, so later cached tensors can still depend on the adapter. The safe general contract is to include adapter provenance in request and prefix-cache identity, then permit reuse only when end-to-end equivalence is explicitly proven.

This connects three schedulers that are often designed separately:

token scheduler:
    which prefill/decode rows run now?

adapter residency policy:
    are the required factors resident and pinned?

KV allocator:
    can request history grow without colliding with adapter memory?

0x01F Runtime State Contract

A request needs at least:

request_id
base_model_id
base_revision
adapter_id
adapter_revision
adapter_method
target_modules
rank_by_module
scaling_by_module
adapter_dtype
quantization compatibility
KV cache handle
sampling state

The worker needs:

base model shards
adapter registry
GPU-resident adapter pages
host-resident adapter store
adapter kernel metadata
KV page allocator
active batch composition
in-flight adapter references

Correctness invariant:

Every forward row must use the adapter state and KV history associated with that request at that position.

An adapter is small compared with the base model, but selecting the wrong adapter is a semantic corruption just like reading another request’s KV pages.

0x01G Immutable Runtime Manifest: Identity Before Residency

The field list above is necessary but not yet a contract. A path can be overwritten. A human revision label can be reused. An integer adapter ID can identify a catalog entry without identifying the bytes behind that entry. A GPU slot can keep the same index while its contents change. Two ranks can load shape-compatible tensors that implement different global updates.

That is why a multi-adapter runtime needs several identities rather than one overloaded adapter_id:

Record	Answers	Must remain distinct from
base execution ABI	Which exact base weights, configuration, transforms, and target-module map define the shared operator?	model name or deployment URL
adapter artifact identity	Which method, module specifications, scales, and tensor bytes define the learned delta?	catalog alias, file path, or integer slot ID
adapter semantic ABI	Which base-plus-adapter mathematical function is requested?	the physical layout chosen by one worker
local kernel ABI	Which converted tensor orientation, dtype, packing, and local shard may this kernel consume?	the global adapter artifact
request binding	Which request attempt selected which semantic base/adapter contract?	reusable prefix identity or scheduler row index
K/V and prefix binding	Under which model-state and exact input contract were persistent rows produced?	request ownership, residency, or sampling policy
slot binding	Which worker slot generation and pointer-table generation currently publish the adapter?	semantic adapter identity
generation snapshot	Which catalog, request, slot, batch plan, and cache snapshots were observed together?	one universal mutable counter

The distinction is operational. Reloading the same adapter bytes into another slot changes physical execution ownership but not the mathematical prefix identity. Replacing bytes behind the same adapter name changes semantic identity even if the path, slot, tensor shapes, and rank stay unchanged.

Deriving Artifact And Execution Identities

Let canonical mean a deterministic serialization with sorted object keys, stable module order, explicit integer and string types, and no pointer or path values. The base execution ABI is:

$$H_{base} = \operatorname{SHA256}\!\left( \operatorname{canonical} \left\{ H_{weights}, H_{config}, H_{transforms}, H_{target\_map} \right\} \right).$$

Renderer-safe fallback:

H_base = SHA256(canonical({
    model_weights_sha256,
    model_config_sha256,
    weight_transform_sha256,
    target_module_map_sha256,
}))

The tokenizer and chat template remain input provenance in M5.1. They do not change a base projection once exact token IDs and side inputs are fixed, so they are not silently folded into this weight ABI.

For adapter module $m$, freeze the logical specification:

$$A_m\in\mathbb{R}^{r_m\times d_{in,m}}, \qquad B_m\in\mathbb{R}^{d_{out,m}\times r_m}, \qquad s_m=\frac{\alpha_m}{r_m}.$$

The module record includes its canonical base-module name, input/output dimensions, rank, scaling rule, logical axes, and hashes of the factor bytes. It is not enough to record one global rank: rank can differ by module, and two rank-$r$ tensors can target different projections.

module_spec[m]:
    canonical base-module name
    target role
    base weight shape [d_out, d_in]
    rank r_m
    alpha numerator and scaling rule
    logical A axes [rank, input]
    logical B axes [output, rank]
    A tensor content hash
    B tensor content hash

The immutable adapter artifact identity is then:

$$H_{artifact} = \operatorname{SHA256}\!\left( \operatorname{canonical} \left\{ method, H_{peft\_config}, H_{base}, (module\_spec_m)_{m=1}^{M} \right\} \right).$$

The ordered module sequence is canonicalized by module name before hashing. The catalog namespace, display name, URI, and human revision are recorded for provenance and authorization, but the bytes and interpretation define H_artifact. A registry may map several aliases to the same artifact or one alias to a newer artifact; neither mapping changes content identity.

The global mathematical function requested by a client is represented by:

H_adapter_semantic = SHA256(canonical({
    H_base,
    H_artifact,
    activation_policy_sha256,
}))

Ordinary always-on LoRA can use a fixed activation-policy digest. A method with an invocation boundary or request-dependent activation needs a different digest even if its factor files match.

Logical Factors Are Not Physical Kernel Layouts

The artifact uses the mathematical orientation above. A kernel may not. In the pinned Punica implementation, LoraWeight.wa is stored as [layer, rank, input], while wb is stored as [layer, rank, output] for its expand path even though the logical $B$ matrix is [output, rank]. BatchedLoraWeight then publishes pointer arrays for those converted objects. This is one official implementation trace, not a universal storage convention.

The local kernel ABI must therefore bind the semantic artifact to its conversion and shard:

$$H_{kernel,local} = \operatorname{SHA256}\!\left( \operatorname{canonical} \left\{ H_{adapter\_semantic}, kernel\_family, conversion\_revision, dtype, packing, local\_module\_layouts, parallel\_shard \right\} \right).$$

For a column-parallel base projection with two tensor-parallel ranks, one possible local contract is:

global logical update:
    A_m [r_m, d_in]       replicated
    B_m [d_out, r_m]      sharded over output

rank q local kernel storage:
    A_q [r_m, d_in]
    B_q [r_m, d_out / 2]  physical [rank, output_shard]
    output range [q * d_out/2, (q + 1) * d_out/2)

This does not claim that every runtime must replicate $A$ or store $B$ in that orientation. It states that whichever choice is made must be explicit and must reproduce the global update. Loop 6 compares the placement alternatives and communication consequences; Loop 2 freezes the fields those comparisons are allowed to vary.

Request, Slot, K/V, And Prefix Projections

The request binds semantic state:

request_binding = SHA256(canonical({
    request_namespace,
    request_id,
    attempt,
    lineage_id,
    H_base,
    H_adapter_semantic,
    generation_policy_sha256,
}))

The request ID and attempt prevent a delayed completion from being published into another attempt. The generation policy belongs to request replay and sampling-state ownership. It does not by itself change K/V already computed for an exact prefix.

M6.5 exports H_adapter_semantic to the M5.1 cache manifest as part of its adapter/weight-transform identity. M5.1 then derives the K/V ABI from the full model, representation, position, visibility, and local shard contract, and M5.3 owns the concrete prefix directory and block keys. The safe composition is:

M6.5 adapter semantic ABI
    -> M5.1 artifact_identity.adapter_set_sha256
    -> M5.1 cache_abi_sha256
    -> M5.1/M5.3 prefix_identity_sha256

For exact prefix reuse:

$$H_{prefix} = \operatorname{SHA256}\!\left( \operatorname{canonical} \left\{ H_{cache\_abi}, H_{prompt\_tokens}, H_{side\_inputs}, H_{isolation\_salt} \right\} \right).$$

The prefix key deliberately excludes:

request_id and request attempt:
    ownership is not mathematical compatibility

adapter slot index and slot generation:
    physical residency can change without changing model semantics

sampling policy and RNG offset:
    they choose future tokens, not K/V for an already exact prefix

catalog path or human adapter name:
    aliases are not content identity

It includes the adapter semantic ABI through the cache ABI. A target-module name heuristic is not a replacement: an upstream adapted projection can change hidden states that later layers project into K/V.

A physical launch needs a different binding:

slot_binding = SHA256(canonical({
    worker_epoch,
    device_id,
    slot_index,
    slot_generation,
    pointer_table_generation,
    H_adapter_semantic,
    H_kernel_local,
}))

The scheduler snapshots this binding when it constructs kernel metadata. A late launch with the right adapter name but an old slot generation must fail. A reload of the same artifact into a new slot generation invalidates stale pointers but does not invalidate a semantic prefix produced under the same cache ABI.

Generation Is A Vector, Not One Counter

Several independent generations advance for different reasons:

Generation	Advances when	Protects against
catalog revision	an alias or authorization record is republished	resolving a name to an unintended artifact
request attempt	a request is retried or replayed as a new attempt	delayed completion from an older attempt
slot generation	a physical adapter slot is republished	use-after-replace of resident factors
pointer-table generation	kernel pointer/offset metadata is rebuilt	stale addresses after compaction or replacement
batch-plan generation	row-to-request/adapter segmentation is republished	executing an old row map with a new active batch
cache snapshot generation	committed K/V state advances or rolls back	stale append, rollback, or publication
RNG snapshot generation	sampling state advances	replaying or skipping random draws

There is no meaningful comparison such as slot_generation >= cache_generation. They live in different namespaces. A launch snapshot records the exact tuple it observed and validates each component against its owner before publication.

def validate_launch(row, request, adapter, slot, cache, plan):
    assert row.request_binding == request.binding_sha256
    assert request.base_abi == adapter.base_abi
    assert request.adapter_semantic_abi == adapter.semantic_abi

    assert row.slot_binding == slot.binding_sha256
    assert slot.adapter_semantic_abi == adapter.semantic_abi
    assert slot.local_kernel_abi == row.local_kernel_abi

    assert row.plan_generation == plan.generation
    assert row.cache_snapshot_generation == cache.generation
    assert cache.adapter_semantic_abi == adapter.semantic_abi

These assertions are a correctness oracle, not a proposed hot-path Python implementation. A production engine can use packed integers and generation arrays as long as those compact fields reconstruct the same comparisons.

Worked Rank-And-Shard Identity Trace

Consider a synthetic adapter over two [8, 8] column-parallel projections:

q_proj: rank 2, alpha 4,  A [2,8], B [8,2]
v_proj: rank 4, alpha 8,  A [4,8], B [8,4]
dtype:  2 bytes/scalar
TP:     2 ranks; A replicated, B output-sharded

The global logical artifact contains:

$$N_{adapter} = 2(8+8)+4(8+8) =96\ \text{scalars},$$

or $192$ bytes. Each TP rank stores all of $A$ but half of each $B$:

$$N_{local} = (2\times8+4\times2) + (4\times8+4\times4) =72\ \text{scalars},$$

or $144$ bytes per rank and $288$ bytes across two ranks. The cluster residency exceeds the unique artifact payload because the $A$ factors are replicated. This is an analytical fixture, not a measured allocator total; alignment, page rounding, pointer tables, and workspace are deferred.

Now bind request req-17, attempt 0, to the artifact and publish it in worker slot 3, slot generation 11. A reload of identical bytes into slot generation 12 produces:

same H_artifact
same H_adapter_semantic
same cache ABI and eligible semantic prefix identity
different slot_binding
different pointer-table and launch snapshot requirements

Changing only the v_proj rank from $4$ to $2$, changing one factor byte, or retargeting it to k_proj changes H_artifact, therefore the semantic ABI and cache ABI. Keeping the alias customer-blue does not preserve compatibility.

Executable Manifest And Negative Fixtures

The article-local oracle serializes this synthetic example and rejects single-field mutations. Its negative fixtures include:

base or adapter digest drift
duplicate or unsorted module records
rank, alpha, logical-shape, or tensor-hash drift
local factor orientation or TP output-range drift
request-to-adapter mismatch
slot artifact, slot generation, or pointer-table mismatch
cache-to-adapter mismatch
prefix projection drift
cross-namespace generation mismatch
logical versus local residency-ledger drift

The evidence lives in:

docs/numinens-evidence/m6-5-multi-adapter-serving-runtime/
    runtime_identity_manifest.json
    invalid_runtime_identity_fixtures.json
    runtime_identity_manifest.py
    test_runtime_identity_manifest.py

Passing this oracle proves only that the declared identities, tensor shapes, hash projections, generation references, and analytical bytes are internally consistent. It does not load an adapter, allocate GPU memory, execute a segmented kernel, validate an engine API, or measure latency. Those claims belong to later loops.

0x02 One Base Projection, Many Adapter Projections

For request group $i$:

$$Y_i = X_iW^\top + s_iX_iA_i^\top B_i^\top$$

The base matrix $W$ is shared. Adapter factors vary.

If a batch contains $N$ token rows:

X:
    [N, d_in]

adapter_ids:
    [N]

sequence/request metadata:
    maps every row to its owner

The base projection can run as one ordinary GEMM:

$$Y_{\text{base}} = XW^\top$$

The adapter branch is heterogeneous:

$$\Delta Y_j = s_{a_j} X_j A_{a_j}^\top B_{a_j}^\top$$

where $a_j$ is the adapter assigned to row $j$.

0x02A Naive Homogeneous Grouping

A straightforward implementation groups rows by adapter:

def adapter_branch_by_group(X, adapter_ids, registry):
    output = zeros([X.shape[0], registry.d_out])

    for adapter_id, row_ids in group_rows(adapter_ids):
        adapter = registry[adapter_id]
        Xi = X[row_ids]

        hidden = Xi @ adapter.A.T
        delta = hidden @ adapter.B.T
        output[row_ids] = adapter.scale * delta

    return output

This is correct if metadata and scatter order are correct. It can be inefficient when:

many adapters appear in one iteration
each adapter owns only a few rows
decode contributes one row per active request
kernel-launch overhead dominates tiny GEMMs
rank varies across adapters

Continuous batching can create exactly that shape: a large shared base batch and many tiny adapter-specific sub-batches.

0x02B Segmented Heterogeneous Kernels

Punica introduces a CUDA kernel design for batching operations belonging to different LoRA models. Its serving insight is that the base GEMM remains shared while low-rank work needs a multi-tenant primitive. The paper’s operator and the archived v1.1.0 wrapper use contiguous row intervals. A later indexed Triton path in vLLM v0.12.0 uses a stable row-index permutation instead. Both are segmented execution, but their metadata and ownership contracts are not interchangeable.

For a linear layer with row segment $X_a\in\mathbb{R}^{N_a\times d_{in}}$ assigned to adapter $a$:

$$U_a = X_aA_a^\top \in\mathbb{R}^{N_a\times r_a}$$ $$\Delta Y_a = s_aU_aB_a^\top \in\mathbb{R}^{N_a\times d_{out}}$$

Punica names the heterogeneous primitive SGMV, Segmented Gather Matrix-Vector multiplication. The low-rank branch is two SGMV launches:

SGMV-shrink:
    [N_a,d_in] x [d_in,r_a] -> [N_a,r_a]

SGMV-expand:
    [N_a,r_a] x [r_a,d_out] -> [N_a,d_out]

Source: Punica v1, Figure 3. SGMV groups request segments by adapter-specific weight pointers while preserving their output row ranges. This is Punica’s kernel contract, not a generic name for every heterogeneous LoRA kernel.

“Matrix-vector” describes the decode-heavy thin-row regime; the segmented operator handles multiple rows and adapters. The segment table determines which LoRA weight pointer each row range uses.

A generic segmented runtime needs enough information to identify:

original rows and active adapter rows
either contiguous row segments or indexed row groups
adapter-specific A/B pointers
fixed-rank kernel shape for this Loop 3 contract
semantic scale placement
the output row updated by every grouped row

That list is intentionally broader than one kernel API. The pinned Punica add_lora_sgmv_cutlass wrapper receives two pointer vectors, an int32 offset vector, one layer index, and one scalar rank:

wa_ptr: [S] int64
wb_ptr: [S] int64
offsets: [S + 1] int32, offsets[0] = 0, offsets[-1] = B
lora_rank: one scalar for the whole call

Y[offsets[i]:offsets[i+1]] +=
    X[offsets[i]:offsets[i+1]]
    @ deref(wa_ptr[i])
    @ deref(wb_ptr[i])

The wrapper does not receive arbitrary gather indices, an inverse permutation, a rank table, or a per-adapter scale table. The caller must already present rows in contiguous segment order. By contrast, vLLM v0.12.0 constructs token_indices_sorted_by_lora_ids, active_lora_ids, num_tokens_per_lora, and lora_token_start_loc; its Triton shrink and expand kernels load the original row indices from that sorted-index vector. That indexed path can preserve the original physical input/output layout without first materializing a packed X_grouped tensor.

The common semantic invariants are:

row j reads only adapter a_j
every active row is represented exactly once
base-only rows contribute zero adapter delta
fixed rank and factor orientation match the selected kernel ABI
segment offsets and pointer-table lengths agree
output row j is restored to request order
padding cannot contribute numerically
the adapter delta is added to, not substituted for, the base output

The exact Punica kernel primitive and current framework implementations are version-specific. The stable systems idea is to batch heterogeneous low-rank operations rather than launch one tiny GEMM pair per adapter.

Three implementation families expose the tradeoff:

Adapter branch	Weight movement	Launch pattern	Failure at high adapter cardinality
Python/host loop	reads each adapter in place	two or more launches per adapter	launch serialization and tiny GEMMs
gather + batched matmul	materializes or gathers per-row weights	gather plus BMM for each factor	duplicated weight traffic and temporary buffers
contiguous segmented kernel	follows offsets and one factor pointer per contiguous segment	one shrink and one expand launch across segments	caller must first construct a compatible row order
indexed segmented kernel	follows stable grouped row indices and active-adapter metadata	shrink and expand cover multiple adapter groups	indirect row access and launch-grid waste can matter

Punica’s paper compares SGMV with loop and Gather-BMM baselines and identifies the extra gather/materialization I/O as a central cost. Those measurements are source-reported for its kernels and hardware; the architectural conclusion is the adapter pointer table should be consumed as close to the GEMM as possible.

Punica’s paper constructs segment indices after arranging the batch and says that consecutive requests sharing one LoRA model are grouped when possible. The pinned model wrapper forms the offset vector as a cumulative sum of per-adapter row lengths. This is the precise reason that segment metadata must be consistent with row ordering:

segment_offsets = [0, n0, n0+n1, ...]
adapter_ids = [adapter_for_segment_0, adapter_for_segment_1, ...]

for s in range(num_segments):
    rows = X[segment_offsets[s]:segment_offsets[s + 1]]
    adapter = adapter_table[adapter_ids[s]]
    # SGMV handles these segments in one device kernel, not this host loop.

If continuous batching interleaves rows from the same adapter, the runtime must either sort/group rows and retain the mapping back to their owners, form multiple contiguous segments that may repeat a pointer, or select an indexed kernel. A contiguous segment-offset kernel cannot silently consume arbitrary row order.

0x02C Five Row Coordinates Must Not Be Collapsed

The word “row” is overloaded in a serving runtime. The following coordinates can happen to agree in a homogeneous batch, but they are different objects:

Coordinate	Symbol	Produced by	Consumed by	Typical failure if confused
scheduler row	$j\in[0,N)$	continuous-batch planner	base GEMM and request bookkeeping	result is published to the wrong request
request/sequence owner	$q_j$	request and attention metadata	attention mask, K/V append, logits ownership	adapter grouping is mistaken for an attention boundary
active-adapter row	$c\in[0,N_a)$	base-only compaction	low-rank planner	a base-only row receives an update or an active row disappears
segment-local row	$u\in[o_t,o_{t+1})$	stable grouping or indexed metadata	shrink and expand kernels	row reads the wrong factor pointer
output row	$j=g_u$	scatter map or indexed kernel	additive projection output	numerically correct delta lands in the wrong row

Request boundaries and adapter segments answer different questions. Two requests can use the same adapter and share one adapter segment while retaining different attention/K/V ownership. One request’s prefill rows can also be split by a planner without changing its adapter identity. M6.5 therefore forbids deriving attention visibility from the adapter permutation.

Let the scheduler-order input be

$$X\in\mathbb{R}^{N\times d_{in}}, \qquad a_j\in\{0,1,\ldots,A\},$$

where adapter ID zero denotes the base-only path in this article’s fixture. Define the active set and its stable adapter-grouped permutation:

$$\mathcal I = \{j\mid a_j\neq 0\}, \qquad g = \operatorname{StableSort}_{j\in\mathcal I}(a_j).$$

The gather matrix $G\in\{0,1\}^{N_a\times N}$ is a useful derivation device:

$$G_{u,j}=\mathbf 1[g_u=j], \qquad X_g=GX.$$

No production kernel needs to materialize $G$. It makes two correctness conditions visible: each row of $G$ contains one 1, and each active scheduler row appears in exactly one column selected by $G$. Those conditions are the injectivity and coverage checks in the executable oracle.

The Loop 3 fixture uses:

N = 8, d_in = 4, r = 2, d_out = 3
adapter IDs in scheduler order:
    [1, 2, 1, 0, 2, 2, 1, 0]

active rows in scheduler order:
    [0, 1, 2, 4, 5, 6]

stable adapter-grouped permutation g:
    [0, 2, 6, 1, 4, 5]

segment adapter IDs:
    [1, 2]

segment offsets o:
    [0, 3, 6]

The original batch is deliberately not adapter-contiguous:

Scheduler row $j$	Request	Adapter $a_j$	Compact/grouped position	Segment
0	req-a	1	0	adapter 1
1	req-b	2	3	adapter 2
2	req-a	1	1	adapter 1
3	req-c	0	—	base only
4	req-b	2	4	adapter 2
5	req-d	2	5	adapter 2
6	req-e	1	2	adapter 1
7	req-c	0	—	base only

Notice that rows 0 and 2 belong to the same request, but row 6 does not; all three still share adapter segment 1. Conversely, base-only rows 3 and 7 share a request but never enter the compact low-rank path.

0x02D From Row Ownership To CSR-Like Segment Metadata

For segment $t$, define adapter ID $c_t$ and half-open offsets $[o_t,o_{t+1})$. The segment contract is:

$$\forall u\in[o_t,o_{t+1}): \quad a_{g_u}=c_t.$$

The metadata resembles compressed sparse row indirection:

grouped row indices g:   [0, 2, 6, 1, 4, 5]
segment offsets o:       [0,       3,       6]
segment adapter IDs c:   [1,       2]
A pointer labels:        [A1,      A2]
B pointer labels:        [B1,      B2]

Its invariants are stronger than “the lengths look plausible”:

o[0] == 0
o[-1] == len(g) == N_active
o[t] < o[t+1] for this no-empty-segment contract
len(c) == len(A_ptr) == len(B_ptr) == len(o) - 1
g is injective, in bounds, and covers exactly the active rows
all rows in segment t have adapter ID c[t]
every pointer label resolves under the admitted local-kernel ABI
launch generations equal the admitted slot, pointer-table, and plan generations

There are two valid physical realizations.

Contiguous interval realization. Materialize or schedule X_grouped = X[g], call a Punica-like operator with offsets [0,3,6], then scatter the packed result through g. The pinned Punica wrapper itself only sees contiguous intervals and pointers; the gather/scatter is an outer runtime responsibility if scheduler order was not already compatible.

Indexed realization. Leave $X$ and $Y$ in scheduler order. Pass g, adapter counts, start locations, and active adapter IDs to a kernel that uses the indexed scheduler rows as its row-address vector. vLLM v0.12.0 follows this family. Its metadata preparation uses a stable sort over token LoRA IDs. It includes a -1 no-LoRA sentinel group rather than compacting those rows:

token_lora_mapping:
    [1, 2, 1, -1, 2, 2, 1, -1]

stable sorted row indices:
    [3, 7, 0, 2, 6, 1, 4, 5]

active_lora_ids:
    [-1, 1, 2]

num_tokens_per_lora:
    [2, 3, 3]

lora_token_start_loc:
    [0, 2, 5, 8]

The pinned shrink and expand kernels read row addresses from the sorted-index vector and early-exit a launch slice whose LoRA ID is -1. “Compact away base-only rows” and “retain a sentinel segment” are therefore alternative metadata encodings, not different LoRA mathematics.

0x02E Exact Shrink, Expand, And Scatter Trace

The fixture uses logical factor axes:

$$A_a\in\mathbb{R}^{r\times d_{in}}, \qquad B_a\in\mathbb{R}^{d_{out}\times r}.$$

Its adapter factors and semantic scales are:

$$A_1= \begin{bmatrix} 1&0&1&0\\ 0&1&0&1 \end{bmatrix}, \quad B_1= \begin{bmatrix} 1&0\\ 0&1\\ 1&1 \end{bmatrix}, \quad s_1=\frac{1}{2},$$ $$A_2= \begin{bmatrix} 1&1&0&0\\ 0&0&1&1 \end{bmatrix}, \quad B_2= \begin{bmatrix} 1&1\\ 1&-1\\ 0&1 \end{bmatrix}, \quad s_2=2.$$

After gathering active rows through $g$, the packed input is:

$$X_g= \begin{bmatrix} 1&0&1&0\\ 1&1&0&0\\ 1&1&1&1\\ 0&1&0&1\\ 0&1&1&0\\ 1&0&0&1 \end{bmatrix}.$$

The fixed-rank shrink stage applies one $A$ pointer to each interval:

$$Z_{u,:} = X_{g_u,:}A_{c_t}^{\top}, \qquad u\in[o_t,o_{t+1}),$$

which produces:

$$Z= \begin{bmatrix} 2&0\\ 1&1\\ 2&2\\ 1&1\\ 1&1\\ 1&1 \end{bmatrix} \in\mathbb{R}^{6\times2}.$$

The expand stage applies the corresponding $B$ pointer and semantic scale:

$$D_{u,:} = s_{c_t}Z_{u,:}B_{c_t}^{\top}, \qquad u\in[o_t,o_{t+1}),$$

so:

$$D= \begin{bmatrix} 1&0&1\\ \frac12&\frac12&1\\ 1&1&2\\ 4&0&2\\ 4&0&2\\ 4&0&2 \end{bmatrix} \in\mathbb{R}^{6\times3}.$$

Two row calculations expose both scales. Scheduler row 2 uses adapter 1:

$$[1,1,0,0]A_1^\top=[1,1],$$ $$\frac12[1,1]B_1^\top = \left[\frac12,\frac12,1\right].$$

Scheduler row 1 uses adapter 2:

$$[0,1,0,1]A_2^\top=[1,1],$$ $$2[1,1]B_2^\top=[4,0,2].$$

Scatter is not optional bookkeeping. Define the scheduler-order delta as zeros and add each packed row back through $g$:

$$\Delta Y_{g_u,:} \mathrel{+}=D_{u,:}.$$

The result is:

$$\Delta Y= \begin{bmatrix} 1&0&1\\ 4&0&2\\ \frac12&\frac12&1\\ 0&0&0\\ 4&0&2\\ 4&0&2\\ 1&1&2\\ 0&0&0 \end{bmatrix}.$$

Rows 3 and 7 remain exactly zero in the adapter branch. For the fixture’s base matrix $W$, the final additive output is:

$$Y=XW^\top+\Delta Y = \begin{bmatrix} 2&1&2\\ 5&1&3\\ \frac32&\frac32&3\\ 3&0&2\\ 4&2&3\\ 6&0&3\\ 3&3&4\\ 1&2&0 \end{bmatrix}.$$

The visual and matrices expose the same contract. The source-reported Punica figure explains the operator family; this program’s own trace proves the row coordinates, exact shapes, and scatter relation for one synthetic case.

0x02F Scale Placement And Physical Factor Layout

The semantic update is invariant under several exact-arithmetic factorizations:

$$s_a(XA_a^\top)B_a^\top = X(s_aA_a)^\top B_a^\top = XA_a^\top(s_aB_a)^\top.$$

That equality does not make scale placement an ignorable runtime detail. It affects artifact hashes, converted tensor bytes, kernel arguments, accumulator range, rounding order, and hotswap compatibility. A manifest must record whether the scale is:

passed as a launch scalar
stored per adapter in a table
folded into A
folded into B
applied by an outer epilogue

The pinned implementations illustrate why versioned code traces matter:

Pinned path	Logical/physical factor surface	Scale surface	Row-address surface
Punica add_lora_bgmv	stacked A [*,L,R,H1], B [*,L,H2,R]	one scalar argument	one weight index per row
Punica add_lora_sgmv_cutlass	pointer target A [L,H1,R], B [L,R,H2]	no scale argument	contiguous offsets [S+1]
Punica custom shrink + CUTLASS expand	stored A [L,R,H1], B [L,R,H2]; shrink interprets A.T	no scale argument	contiguous offsets [S+1]
vLLM v0.12.0 Triton shrink/expand	stacked adapter factors plus row-index metadata	one scalar on shrink	stable sorted original-row indices

Therefore the conceptual scale_table from an abstract heterogeneous API must not be projected onto the pinned Punica SGMV wrapper. For the fixture’s per-adapter scales, a compatible contiguous implementation could fold each scale into its $B_a$ object before publishing the pointer table. The exact oracle proves that this preserves rational outputs. It does not claim bitwise identity in FP16/BF16 because changing scale placement can change rounding and overflow behavior.

Likewise, the pinned Punica wrapper carries one lora_rank scalar and its model integration takes the rank from the first adapter. Loop 3 therefore uses one rank for both adapters. Different ranks require padding, bucketed launches, rank-specialized kernels, or a genuinely heterogeneous rank ABI; Loop 4 owns that comparison.

0x02G Executable Equivalence Oracle

The canonical planner can be written independently of a CUDA API:

def build_fixed_rank_plan(row_adapter_ids, base_id, registry):
    active_rows = [
        row for row, adapter_id in enumerate(row_adapter_ids)
        if adapter_id != base_id
    ]
    permutation = sorted(
        active_rows,
        key=lambda row: row_adapter_ids[row],
    )  # stable

    segment_adapter_ids = []
    offsets = [0]
    for grouped_row, original_row in enumerate(permutation):
        adapter_id = row_adapter_ids[original_row]
        if not segment_adapter_ids or adapter_id != segment_adapter_ids[-1]:
            if segment_adapter_ids:
                offsets.append(grouped_row)
            segment_adapter_ids.append(adapter_id)
    offsets.append(len(permutation))

    assert all(registry[a].rank == registry[segment_adapter_ids[0]].rank
               for a in segment_adapter_ids)
    return permutation, segment_adapter_ids, offsets

The contiguous reference makes gather and scatter explicit:

def contiguous_segmented_lora(X, plan, registry, d_out):
    grouped_X = X[plan.permutation]
    grouped_delta = zeros([len(plan.permutation), d_out])

    for segment, adapter_id in enumerate(plan.segment_adapter_ids):
        lo = plan.offsets[segment]
        hi = plan.offsets[segment + 1]
        adapter = registry[adapter_id]

        Z = grouped_X[lo:hi] @ adapter.A.T
        grouped_delta[lo:hi] = adapter.scale * (Z @ adapter.B.T)

    delta = zeros([X.shape[0], d_out])
    delta[plan.permutation] += grouped_delta
    return delta

An indexed kernel implements the same equations while reading and writing plan.permutation rows directly. It need not allocate grouped_X or execute the Python loop. The pseudocode is an oracle, not a proposed fast path.

The repository evidence bundle freezes the fixture and its failure surface:

docs/numinens-evidence/m6-5-multi-adapter-serving-runtime/
    segmented_tensor_fixture.json
    invalid_segmented_tensor_fixtures.json
    segmented_tensor_oracle.py
    test_segmented_tensor_oracle.py

It uses exact rational arithmetic and proves:

per-row reference == host-grouped reference == segmented reference
split contiguous segments with repeated pointers preserve output
folding each semantic scale into B preserves exact output
reordering scheduler rows and rebuilding metadata preserves row-UID semantics
base-only rows receive an exactly zero adapter delta
all 46 malformed metadata/tensor/generation fixtures fail closed

It also records a hard claim ceiling. Passing the oracle does not execute CUDA or Triton, validate the Punica or vLLM wrappers, measure performance, or prove a production scheduler safe.

0x02H Execution Alternatives And Dispatch Boundary

“Segmented” is not one universally best kernel. The planner should select an execution family under an explicit compatibility and cost contract:

Family	Row addressing	Weight representation	Attractive regime	Main cost or risk
per-row BGMV/indexed	one adapter index per row	stacked same-shape weights	one-row decode, modest active adapter set	repeated pointer/weight traffic and fixed shape requirements
host loop by adapter	host or graph launches one group at a time	ordinary factors	few adapters with substantial rows each	launch serialization and tiny GEMMs
gather + BMM	materialized per-row or per-group weights	dense gathered temporaries	framework portability and easy reference implementation	duplicated weight traffic and temporary memory
contiguous SGMV	CSR-like offsets and pointers	one pointer per segment	planner can make adapter rows contiguous	gather/scatter or scheduling constraints move outside kernel
indexed segmented Triton	stable grouped row indices	stacked/pointer factors	interleaved rows without packed input materialization	indirect access, launch-grid waste, backend-specific limits

The choice depends on row count, adapter cardinality, rank, target projection, dtype, graph-capture constraints, and how much metadata changes each iteration. It cannot be inferred from adapter parameter count alone.

Failure boundaries for this Loop 3 contract are mechanical:

Fault	Shape may still look valid	Consequence	Required response
duplicate or missing row in g	yes	double update or silent omission	reject plan before launch
base-only row included under adapter pointer	yes	wrong model semantics	reject mapping
offset/pointer count mismatch	sometimes	wrong pointer or out-of-bounds interval	reject launch metadata
stale pointer-table generation	yes	use-after-evict or wrong adapter object	fence and rebuild plan
logical/physical factor orientation mismatch	often	wrong numbers with legal GEMM shapes	reject local kernel ABI
scale applied twice or not applied	yes	systematic output error	bind scale placement to artifact conversion
grouped result not scattered	yes	correct values assigned to wrong requests	fail row-UID reconciliation
adapter grouping reused as attention grouping	yes	cross-request visibility or K/V corruption	keep attention metadata independent

This closes the fixed-rank tensor contract. It does not yet choose how one iteration combines prefill and decode rows, nor how different ranks coexist. Those are the owner questions of Loop 4 and Sections 0x03/0x06.

0x03 Prefill And Decode Have Different Adapter Shapes

The same request changes operator shape over its lifetime. During prefill, one request contributes a sequence of token rows. During ordinary one-token decode, it contributes one row to the current iteration:

$$X_u^{\mathrm{pre}} \in \mathbb{R}^{S_u \times d_{in}}, \qquad X_v^{\mathrm{dec}} \in \mathbb{R}^{1 \times d_{in}}.$$

The low-rank branch is therefore matrix-matrix work in prefill and approaches matrix-vector work in decode:

$$Z_u^{\mathrm{pre}} = X_u^{\mathrm{pre}} A_{a(u)}^\top, \qquad z_v^{\mathrm{dec}} = x_v^{\mathrm{dec}} A_{a(v)}^\top.$$

This is not merely a performance distinction. A runtime must expand one request-level adapter identity to every token row in that request’s prefill, while assigning exactly one current row during one-token decode.

0x03A Three Coordinate Systems Define A Mixed Iteration

Let the iteration admit $P$ prefill requests and $D$ decode requests. If prefill request $u$ contributes $S_u$ rows, then:

$$T_{\mathrm{pre}} = \sum_{u=0}^{P-1} S_u, \qquad T = T_{\mathrm{pre}} + D.$$

A Punica-style phase-major packing puts all prefill rows before all decode rows. Prefill request boundaries are represented by an exclusive prefix sum:

$$p_0=0, \qquad p_{u+1}=p_u+S_u, \qquad p_P=T_{\mathrm{pre}}.$$

The first decode row is therefore at doff = T_pre, and decode request $v$ owns row $T_{\mathrm{pre}}+v$. These coordinates coexist:

Coordinate	Question answered	May be shared?
request/sequence interval $[p_u,p_{u+1})$	which rows may attend within one prefill sequence?	never across requests
phase interval $[0,T_{\mathrm{pre}})$ or $[T_{\mathrm{pre}},T)$	which attention kernel and K/V action apply?	fixed for one launch plan
adapter run or indexed group	which $A$, $B$, rank, and scale apply to a row?	may cross the phase boundary

The last row is the subtle case. Two adjacent rows can share an adapter while belonging to different requests and different attention phases. Adapter coalescing does not license attention visibility.

0x03B Exact Five-Request Mixed-Batch Trace

Use five scheduler-order requests:

Scheduler request	Phase	Iteration rows	Adapter	Logical rank
p0	prefill	3	a	2
p1	prefill	2	b	3
d0	decode	1	a	2
d1	decode	1	base only	0
d2	decode	1	c	1

Within each phase, the scheduler may reorder requests if it records the inverse mapping. Put p1 before p0, then append decode rows in scheduler order:

packed request order: [p1, p0, d0, d1, d2]

row:       0   1   2   3   4 | 5    6      7
request:  p1  p1  p0  p0  p0 | d0   d1     d2
phase:   pre pre pre pre pre | dec  dec    dec
adapter:   b   b   a   a   a |  a   base    c
rank:      3   3   2   2   2 |  2    0      1

The phase metadata is:

prefill_indptr = [0, 2, 5]
doff           = 5
decode_count   = 3

The contiguous adapter runs are instead:

segment_adapter = [b, a, base, c]
segment_offsets = [0, 2, 6, 7, 8]

Adapter a deliberately spans rows 2:6: rows 2:5 are the p0 prefill, while row 5 is d0 decode. One segmented projection can reuse a’s factor pointers across that boundary. Attention must still split at doff = 5 and must still respect prefill_indptr = [0,2,5].

0x03C Request Identity Must Expand To Token Identity

Let request $q$ own adapter $a(q)$. The row-level mapping is not a request list; it is an expansion by the number of rows contributed in this iteration:

$$m_i = \begin{cases} a(u), & p_u \le i < p_{u+1}, \\ a(v), & i=T_{\mathrm{pre}}+v. \end{cases}$$

For the fixture:

request adapters in packed order = [b, a, a, base, c]
row adapter mapping             = [b, b, a, a, a, a, base, c]

Incrementing an adapter run once per prefill request instead of once per prefill token produces legal-looking offsets whose final value is the number of requests rather than $T$. The failure is detectable before launch:

$$o_{K}=T$$

for sentinel-inclusive contiguous runs, or:

$$\sum_a N_a=N_{\mathrm{active}}$$

for base-compacted indexed execution.

The archived Punica TUI example is useful source evidence for phase-first, adapter-aware request ordering, but it is not used here as a correctness oracle: its lora_lens construction increments once per request while its prefill input flattens every prompt token. The article therefore derives and tests row expansion independently instead of copying that prototype logic.

0x03D One Dense Projection, Two Attention Paths, Several Adapter Paths

The dense projection can consume the full packed matrix:

$$Y_{base}=XW^\top, \qquad X\in\mathbb{R}^{T\times d_{in}}.$$

The attention layer cannot treat the same matrix as one sequence. A Punica-style model path slices the projected Q/K/V tensors at doff:

qkv = dense_projection_with_adapter_delta(X)

q_pre, k_pre, v_pre = qkv[:doff]
prefill_attention(q_pre, k_pre, v_pre, prefill_indptr)

q_dec, k_dec, v_dec = qkv[doff:]
append_decode_kv(k_dec, v_dec, decode_request_handles)
decode_attention(q_dec, decode_request_handles)

The adapter branch has more implementation choices:

Path	Prefill shape	Decode shape	Rank contract	Metadata contract
request loop	$[S_u,d]$ GEMM	$[1,d]$ GEMV	exact per request	request slices
S-LoRA-style	MBGMM	MBGMV	per-adapter rank plus max-rank workspace	request starts, lengths, rank table, page locations
vLLM v0.12.0-style	indexed grouped rows	indexed grouped rows	resident slots padded to configured max_lora_rank	token mapping, stable sorted indices, counts, starts
pinned Punica SGMV wrapper	contiguous segmented rows	contiguous segmented rows	one fixed rank argument	pointer vector plus offsets
rank buckets	one launch per prefill/rank bucket	one launch per decode/rank bucket	fixed rank inside each bucket	gather/scatter per bucket

These are alternative physical schedules for the same row-wise function. They are not interchangeable ABIs.

0x03E Sampling Rows Need Their Own Inverse Map

The model produces one logit row per packed token, but a normal iteration emits one next-token distribution per request. For prefill request $u$, select its last packed token; for decode request $v$, select its only current row:

$$\phi(u)=p_{u+1}-1, \qquad \phi(v)=T_{\mathrm{pre}}+v.$$

Because the fixture reordered p1 before p0, restoring scheduler request order [p0,p1,d0,d1,d2] requires:

packed sampling rows  = [4, 1, 5, 6, 7]
scheduler output slots = [p0,p1,d0,d1,d2]

The adapter scatter map and sampling-row map are different objects. The former returns every adapter delta to one packed token row; the latter selects and reorders five logit rows for five requests.

0x03F Fragmentation Is Phase-Conditional

For adapter $a$ and phase $\pi\in\{\mathrm{pre},\mathrm{dec}\}$, define:

$$N_{a,\pi} = \left|\{i:m_i=a\land \operatorname{phase}(i)=\pi\}\right|.$$

The ordinary adapter concentration ratio is:

$$\rho_{\mathrm{adapter}} = \frac{\max_a N_a}{\sum_a N_a}, \qquad N_a=N_{a,\mathrm{pre}}+N_{a,\mathrm{dec}}.$$

It can hide phase behavior. A popular adapter may have a useful prefill GEMM while appearing as isolated one-row decode work. Record at least:

distinct adapters per phase and iteration
prefill rows and requests per adapter
decode rows and requests per adapter
contiguous runs before and after legal within-phase reordering
rank histogram by phase
adapter branch time by phase
gather/scatter and metadata-build time
padded-rank work / true-rank work

0x03G Mixed-Batch Construction Contract

def build_mixed_iteration(prefills, decodes):
    # Reordering is allowed only with an explicit request-output inverse map.
    ordered_prefills = adapter_aware_order(prefills)
    ordered_decodes = stable_decode_order(decodes)

    rows = []
    request_ids = []
    phases = []
    adapter_ids = []
    prefill_indptr = [0]

    for req in ordered_prefills:
        for token_position, row in enumerate(req.prefill_rows):
            rows.append(row)
            request_ids.append(req.request_id)
            phases.append("prefill")
            adapter_ids.append(req.adapter_id)
        prefill_indptr.append(len(rows))

    doff = len(rows)
    for req in ordered_decodes:
        rows.append(req.current_row)
        request_ids.append(req.request_id)
        phases.append("decode")
        adapter_ids.append(req.adapter_id)

    assert prefill_indptr[-1] == doff
    assert len(rows) == doff + len(ordered_decodes)
    assert len(rows) == len(request_ids) == len(phases) == len(adapter_ids)

    sample_rows = derive_last_prefill_and_decode_rows(
        prefill_indptr, doff, ordered_prefills, ordered_decodes
    )
    return freeze_launch_plan(rows, request_ids, phases, adapter_ids, sample_rows)

This builder freezes metadata once per model invocation. Every layer consumes the same row ownership. Kernel-local pointer tables may differ by target module, but they must bind to the same batch-plan generation.

0x04 Adapter Memory Is A Residency Problem

For one adapter $a$:

$$M_a = \sum_{m \in \mathcal{T}_a} r_{a,m} (d_{in,m}+d_{out,m}) b_{a,m}$$

where:

T_a:
    target modules in adapter a

r_a,m:
    rank for module m

b_a,m:
    bytes per adapter parameter

Total served memory is not only:

$$M_{\text{base}} + \sum_a M_a$$

It includes:

$$M_{\text{worker}} = M_{\text{base}} + M_{\text{resident adapters}} + M_{\text{KV}} + M_{\text{activations}} + M_{\text{kernel workspaces}} + M_{\text{allocator slack}}$$

Renderer-safe ledger:

M_worker =
    M_base
  + M_resident_adapters
  + M_KV
  + M_activations
  + M_workspaces
  + M_fragmentation

Adapter size may be small enough that thousands fit in host memory while only a subset fit comfortably in GPU memory alongside KV.

0x04A Residency Hierarchy

object store / local disk:
    durable adapter checkpoints

host memory:
    warm adapter cache

pinned host staging:
    transfer-ready pages

GPU memory:
    active or likely-soon adapter factors

kernel pointer table:
    addresses and shape metadata for current iteration

Lifecycle:

registered
    -> host_resident
        -> transfer_pending
            -> gpu_resident
                -> referenced_by_batch
                    -> idle_gpu_resident
                        -> evictable

An adapter can be evicted only when no in-flight batch, queued request with reservation, or asynchronous kernel still references its storage.

0x04B Load-Time Ledger

If adapter $a$ has size $M_a$ and effective host-to-device bandwidth $BW_{h2d}$:

$$T_{\text{load},a} \gtrsim \frac{M_a}{BW_{h2d}} + T_{\text{metadata}} + T_{\text{allocation}} + T_{\text{registration}}$$

Small adapters can still suffer latency from fixed overheads, synchronization, and allocator behavior.

Cold-start TTFT:

$$TTFT_{\text{cold adapter}} \approx T_{\text{queue}} + T_{\text{adapter load}} + T_{\text{prefill}} + T_{\text{first decode}}$$

This is why adapter residency belongs in admission control, not only in a background cache.

0x04C Residency Is A State Machine, Not A Boolean

“Loaded” hides at least six different claims. A checkpoint can be present in host memory while no GPU slot exists. Device pages can be reserved while a DMA copy is incomplete. A copy can be complete while no current pointer table is allowed to name it. A published adapter can be idle but still pinned by a queued reservation or referenced by an asynchronous launch.

Use an explicit object state:

absent
    -> host_ready
        -> reserved
            -> loading
                -> validated
                    -> published
                        -> active
                            -> idle
                                -> retiring
                                    -> absent

The state names are a rewrite contract, not an API copied from S-LoRA or vLLM. They make the safety conditions visible:

State	Device pages assigned?	Pointer-table entry visible?	New batch may bind?
host_ready	no	no	no
reserved	yes	no	no
loading	yes	no	no
validated	yes	no	no
published	yes	yes	yes
active	yes	yes	already bound
idle	yes	yes	yes
retiring	yes until reclaimed	no for new binds	no

Two transitions are especially important:

publication:
    validated bytes + complete page table + immutable adapter identity
    -> one generation-stamped visible mapping

retirement:
    stop new binds
    -> wait for references, pins, reservations, and device events
    -> unpublish
    -> reclaim pages

Publishing an adapter before its final copy or validation completes creates a shape-correct use-after-partial-load bug. Freeing it after removing only a host catalog entry creates a device use-after-free bug.

The pinned vLLM v0.12.0 code provides a useful but different concrete model. It separates registered adapters, bounded by max_cpu_loras, from active GPU slots, bounded by max_loras. Its LRU worker loads and validates a new adapter before removing the oldest registered adapter, while its LRU manager can pin registered and active cache entries. Those implementation facts license the separation of registration, activation, and eviction; they do not prove the transaction protocol derived here.

0x04D Derive Page Demand From Tensors, Not From Rank Alone

Let one primitive vector page hold $H$ elements of $b$ bytes each:

$$B_{\text{vector page}} = H b$$

For a tensor with $E$ elements, a vector-page upper bound is:

$$P(E;H)=\left\lceil\frac{E}{H}\right\rceil$$

Therefore a general adapter page ledger is:

$$P_a = \sum_{m\in\mathcal{T}_a} \left[ P\!\left(r_{a,m}d_{in,m};H\right) + P\!\left(d_{out,m}r_{a,m};H\right) \right]$$

This formula is deliberately more general than “rank $r$ takes $r$ pages.” S-LoRA’s paper observes that one compatible LoRA weight tensor of shape $[R,H]$ occupies $R$ vector pages. A complete adapter can contain two factors, several target modules, and every transformer layer.

The final archived S-LoRA code makes a stronger source-specific packing choice. Its allocator owns key and value buffers for every layer. One allocator index selects one hidden-width row in both buffers at every layer, so the bytes behind one index are:

$$B_{\text{S-LoRA slot}} = 2 L H b$$

where the factor $2$ is the paired key/value storage reused for LoRA $A/B$-like factors, and $L$ is the number of layers. In the pinned attention path, load_adapters reserves 4 * rank such indices for the four (q,k,v,o) projection groups:

$$P_{a,\text{pinned S-LoRA attention}} = 4r_a$$

and the corresponding payload represented by those slots is:

$$M_{a,\text{slot payload}} = 4r_a\,(2LHb)$$

This is a trace of one archived implementation, not a universal LoRA page formula. MLP adapters, unmatched factor widths, quantized factors, tensor parallel shards, and another allocator change the count.

Renderer-safe derivation:

primitive vector page bytes = H * bytes_per_element

general adapter pages =
    sum over target modules (
        ceil(rank * d_in / H)
      + ceil(d_out * rank / H)
    )

pinned S-LoRA paired multi-layer slot bytes =
    2 * layers * H * bytes_per_element

pinned S-LoRA q/k/v/o attention slots =
    4 * rank

0x04E Reserve, Copy, Validate, Then Publish

The load path is a transaction over identity, pages, bytes, and visibility:

def prepare_adapter(txn, artifact, request, pool):
    validate_semantic_and_kernel_abi(artifact, request)
    demand = derive_adapter_and_kv_page_demand(artifact, request)

    victims = choose_eligible_idle_adapters(pool, demand.shortfall)
    reservation = pool.reserve_atomically(demand.total, victims)

    try:
        mark_retiring_and_block_new_pins(victims, txn.generation)
        copy_from_trusted_host_source(artifact, reservation.adapter_pages)
        await_copy_event()
        validate_digest_layout_dtype_and_generation(artifact, reservation)

        # One visibility point: kernels must see either the old table or this
        # complete new table, never a partially written mapping.
        publish_adapter_mapping(
            adapter_id=artifact.id,
            pages=reservation.adapter_pages,
            generation=txn.generation,
        )
        pin_for_request(artifact.id, request.attempt_id)
        commit_kv_reservation(request.attempt_id, reservation.kv_pages)
        finalize_victim_eviction(victims)
        return admission_success(reservation)
    except Exception:
        unpublish_if_visible(artifact.id, txn.generation)
        release_request_pins_and_kv_reservation(request.attempt_id)
        cancel_or_wait_transaction_events(reservation)
        pool.release_nonvictim_pages(reservation)
        restore_retired_victims_from_trusted_host_source(
            victims, reservation.victim_pages
        )
        pool.restore_victim_ownership_atomically(
            victims, reservation.victim_pages
        )
        raise

The pseudocode orders concepts, not CUDA calls. A production implementation must also define stream/event ordering, lock or single-threaded ownership, metadata durability, and what happens if compensating restoration itself fails.

The archived S-LoRA loader allocates non-contiguous indices, extends adapter metadata, and then copies factors, optionally on a prefetch stream. That code is valuable evidence for the physical design, but it does not expose this full transaction protocol. The derived protocol is intentionally stricter.

0x04F Reference, Pin, Event, Reservation, And Generation Counts

An idle adapter is not automatically evictable. Define:

refcount(a):
    admitted or running request attempts bound to adapter a

pin_count(a):
    explicit policy pins, operator pins, or administrative pins

queued_reservations(a):
    accepted queue entries whose admission assumes a remains resident

inflight_events(a):
    device copies or kernels that may still access a's pages

published_generation(a):
    generation encoded in the pointer table used by a launch snapshot

Then an adapter is eligible for retirement only if:

$$\operatorname{evictable}(a) = \mathbf{1}\left[ \begin{aligned} &state_a\in\{\text{published},\text{idle}\}\\ &\land\ refcount(a)=0\\ &\land\ pin\_count(a)=0\\ &\land\ queued\_reservations(a)=0\\ &\land\ inflight\_events(a)=0\\ &\land\ generation_a\notin\mathcal{G}_{\text{live launches}} \end{aligned} \right]$$

LRU, LFU, size-aware, or deadline-aware policy may rank only the objects that pass this predicate. Recency is not a safety proof.

0x05 Unified Paging: Adapter Pages And KV Pages Compete

S-LoRA stores adapters in main memory, fetches active adapters to GPU, and proposes Unified Paging to manage dynamic adapter weights with different ranks together with KV tensors of varying sequence lengths.

Source: S-LoRA v3, Figure 3. S-LoRA places variable-length KV state and variable-rank adapter weights in one paged pool. Sharing a page allocator reduces fragmentation but does not make adapter and KV ownership or lifetimes identical.

The two objects differ:

Object	Size changes when	Lifetime
adapter pages	adapter is loaded, converted, or evicted	shared across requests using that adapter
KV pages	request prefills or decodes	private or prefix-shared under M5 rules

But both compete for the same device memory.

S-LoRA’s page-shape observation is more specific than “put both in one allocator.” For hidden-width-like dimension $h$:

one KV tensor:
    [sequence_length,h]
    -> sequence_length vector pages

one LoRA factor with compatible orientation:
    [rank,h]
    -> rank vector pages

A statically reserved device buffer is divided into vector-sized pages. Page tables reconstruct non-contiguous KV sequences and non-contiguous adapter factors. The allocator can therefore place adapter and KV pages in the same pool without requiring each variable-size object to occupy one contiguous region.

This does not make their lifetimes identical:

adapter page:
    shared by requests using adapter a
    retained or evicted by adapter popularity/residency policy

KV page:
    owned by one request or a valid prefix-sharing relation
    grows as accepted tokens are appended

Reference counts and eviction eligibility must preserve those ownership rules even when the physical page allocator is unified.

S-LoRA’s heterogeneous kernels are coupled to this layout: they operate on non-contiguous adapter pages and varying ranks directly. A fallback that first copies every adapter into a contiguous padded tensor changes the bandwidth, workspace, and fragmentation ledger.

Let:

$$P_{free} = P_{total} -P_{base} -P_{workspace} -P_{adapter} -P_{KV}$$

An admission decision must consider the joint pressure.

Bad independent policies:

adapter cache:
    fills all spare memory with warm adapters

KV allocator:
    assumes spare memory is available for request growth

result:
    decode request cannot append KV
    or adapter load forces emergency KV eviction/preemption

Joint policy sketch:

def can_admit(req, worker):
    adapter_need = worker.adapter_cache.incremental_bytes(req.adapter_id)
    kv_need = estimate_request_kv_bytes(req)
    reserve = worker.decode_growth_reserve

    required = adapter_need + kv_need + reserve
    if required > worker.free_unreserved_bytes:
        return False, "joint_adapter_kv_capacity"

    if not worker.kernel_supports(req.adapter_metadata):
        return False, "adapter_kernel_contract"

    return True, "ok"

That sketch answers only whether enough unreserved bytes appear to exist. A safe implementation must prove which pages are free, which objects may be retired, which generations are visible, and whether every partial transition can be rolled back.

0x05A One Pool Does Not Mean One Ownership Rule

Let the shared pool contain $P_{total}$ allocator slots:

$$P_{total} = P_{free} +P_{reserved} +P_{adapter} +P_{KV}$$

Every slot must occur in exactly one physical state. Object ownership is a second relation:

physical page state:
    free | reserved(txn) | owned(object, generation)

adapter ownership:
    many request attempts may reference one immutable published generation

KV ownership:
    one request attempt or one explicitly valid prefix-sharing relation
    owns each page under M5's cache contract

The common allocator can reduce external fragmentation because both object classes draw non-contiguous slots from one free set. It cannot reuse one slot simultaneously, make adapter pages request-private, or make request KV pages adapter-global.

The byte conservation equation follows from the slot geometry:

$$M_{pool} = P_{total}B_{slot} = (P_{free}+P_{reserved}+P_{adapter}+P_{KV})B_{slot}$$

The equation is an allocator invariant. It excludes base weights, activations, workspace outside the pool, host copies, page tables, and allocator metadata.

0x05B Exact Sixteen-Slot Admission Fixture

Use a deliberately small S-LoRA-compatible paired-slot fixture:

hidden width H:       8 elements
layers L:             2
element bytes b:      2
slot bytes:           2 * L * H * b = 64 bytes
total slots:          16
pool payload:         16 * 64 = 1,024 bytes

initial ownership:
    slots  0..3   KV request r0      active and pinned
    slots  4..7   adapter a          refcount 1, pin_count 1
    slots  8..11  adapter b          idle, unpinned, host-backed
    slots 12..15  free

The incoming request r1 uses a rank-one (q/k/v/o) attention adapter c in the pinned S-LoRA-style layout:

adapter c:              4 * rank = 4 slots = 256 bytes
initial request KV:                       2 slots = 128 bytes
decode-growth reserve:                    2 slots = 128 bytes
total transaction demand:                 8 slots = 512 bytes

free now:                                4 slots = 256 bytes
eligible reclaim from adapter b:          4 slots = 256 bytes
free + reclaimable:                       8 slots = 512 bytes

Adapter b is the only legal victim. Request r0’s KV and adapter a fail the pin/reference predicate even though their pages are physically adjacent.

Successful transaction:

1. validate adapter c identity, target set, rank, dtype, and local kernel ABI
2. derive demand: adapter 4 + initial KV 2 + growth KV 2 = 8 slots
3. prove adapter b is eligible; mark it retiring and block new pins
4. atomically reserve slots 8..15 for transaction tx7
5. copy c into slots 8..11; wait for the copy event; validate digest/layout
6. publish c generation 1 with page table [8,9,10,11]
7. pin c for request attempt r1#1
8. commit r1 KV reservation [12,13,14,15]
9. finalize b's eviction and generation tombstone

final conservation:
    KV r0       4
    adapter a   4
    adapter c   4
    KV r1       4
    total      16

This fixture is intentionally at the capacity boundary. It reveals any policy that double-counts victim pages, forgets the growth reserve, or publishes c before validation.

0x05C Joint Admission Inequality

For request $q$, define:

P_a,incremental(q):
    zero when the exact required adapter generation is already resident;
    otherwise pages needed for its local physical representation

P_KV,initial(q):
    pages needed to admit the prompt/prefill state under M5's rules

P_KV,growth(q):
    reserved decode growth under the configured admission horizon

P_guard(q):
    pool-local safety margin; external kernel workspace is accounted outside

P_reclaimable:
    pages owned only by victims that satisfy the full eviction predicate

Admission requires:

$$P_{free}+P_{reclaimable} \ge P_{a,incremental}(q) +P_{KV,initial}(q) +P_{KV,growth}(q) +P_{guard}(q)$$

and the same comparison in bytes:

$$(P_{free}+P_{reclaimable})B_{slot} \ge P_{demand}(q)B_{slot}$$

The inequality is necessary, not sufficient. It does not prove:

the adapter artifact is valid
the chosen backend supports its physical ABI
the load can finish before the request deadline
the victim can be restored if the transaction fails
the host/device transfer path has capacity
the future request trace fits the growth horizon

M5 owns the algorithm that estimates P_KV,initial and P_KV,growth. M6.5 consumes those numbers and combines them with adapter residency.

0x05D Victim Eligibility, Ranking, And Reuse Are Separate Steps

Eviction has three contracts:

1. Eligibility is a hard safety predicate.
2. Ranking chooses among eligible objects.
3. Reuse decides when old pages may be overwritten.

def eligible(adapter, live_launch_generations):
    return (
        adapter.state in {"published", "idle"}
        and adapter.refcount == 0
        and adapter.pin_count == 0
        and adapter.queued_reservations == 0
        and adapter.inflight_events == 0
        and adapter.generation not in live_launch_generations
        and adapter.host_source_verified
    )

def choose_victims(adapters, pages_needed):
    candidates = [a for a in adapters if eligible(a, live_generations())]
    candidates.sort(key=lambda a: (a.last_touch, -len(a.pages), a.adapter_id))

    victims, reclaimed = [], 0
    for adapter in candidates:
        victims.append(adapter)
        reclaimed += len(adapter.pages)
        if reclaimed >= pages_needed:
            return victims
    raise AdmissionError("INSUFFICIENT_ELIGIBLE_PAGES")

This example ranks by LRU, then prefers a larger object, then uses stable identity as a deterministic tie-break. LFU, cost-aware, deadline-aware, or reuse-aware ranking is also possible. None may bypass eligibility.

Once a victim is marked retiring, new binds must fail or retry against a new generation. Its pages become reusable only after the last event/reference fence. Merely deleting an entry from a Python dictionary is not a device lifetime barrier.

0x05E Publication Is One Atomic Visibility Point

A kernel launch must snapshot a self-consistent tuple:

$$\mathcal{L} = (g_{catalog},g_{slot},g_{pointer},g_{batch},\operatorname{pages}(a))$$

The adapter becomes usable only after:

all reserved pages contain validated bytes
all pointer-table entries name those pages
rank, scale, dtype, target, and shard metadata match the artifact
the copy-complete event happens-before pointer publication
the published generation matches the batch-plan snapshot

Readers may observe either the previous complete mapping or the next complete mapping. They may not observe a new generation with old pages, old metadata with new pages, or a half-written pointer table.

Possible implementations include:

double-buffered pointer tables + generation flip
copy-on-write metadata object + atomic handle publication
single-threaded engine ownership + stream event + generation check
lock-protected table update + launch snapshot fence

The choice is runtime-specific. The invariant is not.

0x05F Partial-Failure Rollback Is Compensating Work

Suppose the copy of adapter c fails after slots 8..11 have overwritten the retired adapter b. Releasing the transaction bitmap is insufficient: the old published bytes no longer exist in those slots.

Required rollback trace:

before failure:
    b retired and unpublished
    slots 8..15 reserved by tx7
    c copy incomplete
    c never published
    r1 KV reservation not committed

rollback:
    cancel/wait tx7 device events
    remove every unpublished c mapping and request pin
    release r1 KV reservation
    reload b from its verified host backing into slots 8..11
    validate b and publish fresh fixture generation b@g5
    return slots 12..15 to free

after rollback:
    r0 KV and adapter a unchanged
    adapter b@g5 resident again under a new generation
    adapter c absent
    request r1 not admitted
    pool conservation restored to 16 slots

This is compensating rollback, not free atomicity. It assumes a trusted host source for b and does additional I/O. A runtime that cannot guarantee that restoration should avoid overwriting victim pages before commit, reserve a shadow region, or reject the admission.

Failure after publication is a different transaction boundary. The runtime must first prevent new binds, drain or abort every request that observed the new generation, then retire it. It cannot silently pretend publication never happened.

0x05G Physical Design Alternatives

Unified vector paging is one design point:

Design	Fragmentation / capacity property	Load and kernel consequence	Failure consequence
unified adapter + KV vector pages	flexible shared capacity; one class can pressure the other	kernels gather non-contiguous adapter and KV rows	joint admission and cross-class starvation control required
separate adapter and KV pools	hard isolation and simpler ownership accounting	stranded capacity if one pool is idle	adapter churn cannot consume KV reserve, but resizing is operational work
fixed max-rank GPU slots	predictable pointer arithmetic and bounded active set	rank padding wastes storage/work; slot count is explicit	activation fails when no legal slot exists
host-warm cache + device LRU	many registered adapters, bounded active set	miss adds load/activation latency	invalid new artifact should be rejected before destructive eviction
load per batch, offload after batch	minimal idle device residency	transfer and synchronization can dominate	simple lifetime, poor reuse under repeated adapters
shadow-copy then publish	strongest pre-publication rollback	needs temporary duplicate capacity	old generation remains intact until commit

The pinned S-LoRA paper/code support the first and fifth rows, including prefetch overlap. The pinned vLLM v0.12.0 manager illustrates separate registered and active capacities plus LRU/pinning behavior. The remaining rows are design alternatives used to expose the contract, not claims that either source implements them exactly.

0x05H Executable Residency Oracle

The checked-in Loop 5 oracle replays the sixteen-slot fixture with exact integer accounting. It verifies:

slot conservation and unique ownership
S-LoRA-compatible slot-byte derivation
incremental adapter + initial KV + growth KV demand
full victim-eligibility predicate
deterministic victim choice
atomic reservation coverage
copy -> validate -> publish ordering
generation-fenced pointer visibility
success-state ownership
copy-failure compensating rollback
digest-failure rollback
negative mutations for double ownership, stale generations, illegal victims,
premature publication, missing host backing, and leaked reservations

It allocates no GPU memory, performs no DMA, executes no CUDA or Triton kernel, and measures no latency or throughput. Its result is a state/page/byte proof, not a production allocator benchmark.

0x06 Rank Heterogeneity Is A Kernel Shape

Two adapters can target the same modules but use different ranks:

adapter A:
    q_proj rank 8

adapter B:
    q_proj rank 64

For one row, branch FLOPs are approximately:

$$F_a \approx 2r_a(d_{in}+d_{out})$$

Rank ratio:

$$\frac{F_B}{F_A} \approx \frac{r_B}{r_A}$$

for identical matrix shapes.

A heterogeneous batch can therefore have per-row adapter work differing by 8x even though every request uses “LoRA.”

0x06A Logical Rank And Physical Rank Are Different Fields

Let $r_a$ be adapter $a$‘s logical rank and let $R$ be the physical rank of a static slot or intermediate. A max-rank implementation can embed the logical factors in zero-padded physical tensors:

$$\bar A_a = \begin{bmatrix} A_a\\ 0_{(R-r_a)\times d_{in}} \end{bmatrix} \in\mathbb{R}^{R\times d_{in}},$$ $$\bar B_a = \begin{bmatrix} B_a & 0_{d_{out}\times(R-r_a)} \end{bmatrix} \in\mathbb{R}^{d_{out}\times R}.$$

The padded update is algebraically identical:

$$\bar B_a\bar A_a = \begin{bmatrix}B_a&0\end{bmatrix} \begin{bmatrix}A_a\\0\end{bmatrix} =B_aA_a.$$

Zero padding is therefore an exact representation transform over exact arithmetic. It is not permission to leave stale values in the padded tails. Both factor tails must be zeroed when a resident slot is reset or overwritten.

The pinned vLLM v0.12.0 linear wrapper follows this family of contracts: it allocates resident A/B slots at configured max_lora_rank, zeros a slot before copying, and copies each logical factor into a leading slice. Its indexed Triton shrink and expand operators then obtain rank from the physical stacked tensor. That is a versioned implementation choice, not a universal vLLM or Punica ABI.

0x06B True-Rank And Padded-Rank Ledgers

For active row $i$ using adapter $a(i)$, the branch’s conceptual multiply-add count is:

$$F_{\mathrm{true}} \approx 2(d_{in}+d_{out}) \sum_{i\in\mathcal{I}_{active}} r_{a(i)}.$$

If every active row executes physical rank $R$:

$$F_{\mathrm{pad}} \approx 2N_{active}R(d_{in}+d_{out}).$$

The arithmetic padding ratio and wasted rank-row count are:

$$\eta_{\mathrm{pad}} = \frac{F_{\mathrm{pad}}}{F_{\mathrm{true}}} = \frac{N_{active}R}{\sum_i r_{a(i)}},$$ $$U_r = N_{active}R-\sum_i r_{a(i)}.$$

For the mixed fixture, seven rows use adapters. Adapter b owns two rank-3 rows, adapter a owns four rank-2 rows, and adapter c owns one rank-1 row:

$$\sum_i r_{a(i)}=2\cdot3+4\cdot2+1\cdot1=15.$$

With $d_{in}=3$, $d_{out}=2$, and $R=3$:

true-rank branch FLOPs   ~= 2 * 15 * (3 + 2) = 150
padded-rank branch FLOPs ~= 2 *  7 * 3 * (3 + 2) = 210
padding ratio             = 210 / 150 = 1.4x
logical rank-row values   = 15
padded intermediate       = 7 * 3 = 21 values
wasted rank-row values    = 6

This ledger counts logical matmul work. It does not predict latency: a padded kernel can still beat several smaller launches, and a ragged kernel can pay indexing, masking, and load-imbalance costs.

0x06C Exact Variable-Rank Tensor Trace

Use these logical factor orientations:

$$A_a\in\mathbb{R}^{r_a\times3}, \qquad B_a\in\mathbb{R}^{2\times r_a}, \qquad D_i=s_{a(i)}(X_iA_{a(i)}^\top)B_{a(i)}^\top.$$

For the eight packed rows from Section 0x03B, the exact ragged shrink rows are:

$$Z_{ragged}= \left[ \begin{array}{c} [1,2,-1]\\ [2,1,1]\\ [2,-1]\\ [1,1]\\ [2,1]\\ [3,-3]\\ []\\ [1] \end{array} \right].$$

The empty row is base-only. After expansion and semantic scale, the adapter delta in packed-row order is:

$$D= \begin{bmatrix} 0&3\\ 3&0\\ 0&-\frac{3}{2}\\ \frac{3}{2}&0\\ 2&-\frac{1}{2}\\ -\frac{3}{2}&-3\\ 0&0\\ 4&2 \end{bmatrix}.$$

The executable Loop 4 oracle computes the same $D$ through four paths:

per-row exact-rank reference
phase-split prefill/decode execution
rank-bucketed gather/compute/scatter
max-rank zero-padded indexed execution

It also reorders the two prefill requests, rebuilds every dependent mapping, and proves equality by immutable row UID and scheduler request ID.

0x06D Runtime Design Alternatives

Strategy	Useful property	Cost or constraint	Fail-closed requirement
pad to configured maximum	one static storage and kernel rank	extra resident bytes, intermediate values, and work	zero both A and B tails; reject $r_a>R$
bucket by rank	exact shapes and predictable kernels	more launches, gathers, scatters, and phase fragmentation	scatter by immutable row UID
fully heterogeneous kernel	one launch can consume per-adapter ranks	rank-aware masking, pointer arithmetic, and uneven work	validate rank table against factor/page extents
exact request loop	simplest semantic oracle	poor launch amortization at high adapter cardinality	preserve packed output mapping
convert at admission	normalize artifacts to supported ranks/layouts	new artifact bytes and possible approximation if rank changes	bind a new local-kernel ABI digest
cap supported rank	bounded workspace and compiled shapes	rejects otherwise valid artifacts	reject before residency/launch

S-LoRA’s paper names separate heterogeneous kernels: MBGMM for prefill sequences and MBGMV for one-token decode. Its pinned code carries per-adapter rank and page-location metadata while allocating an intermediate at the maximum rank in the active adapter set. The pinned Punica wrapper instead takes one rank from the first batched adapter. A mixed-rank Punica invocation must therefore be rejected or partitioned into compatible launches; the paper’s separate rank-8/16/32/64 microbenchmarks do not prove same-launch rank heterogeneity.

0x06E Rank-Aware Dispatch Contract

def choose_adapter_path(batch, backend):
    ranks = sorted({row.logical_rank for row in batch.active_rows})

    if backend.supports_per_adapter_rank_table:
        return heterogeneous_launch(batch, rank_table=ranks)

    if backend.uses_zero_padded_slots:
        if max(ranks, default=0) > backend.max_lora_rank:
            raise AdmissionError("rank_exceeds_physical_slot")
        assert_all_padding_zero(batch.resident_slots)
        return padded_indexed_launch(batch, physical_rank=backend.max_lora_rank)

    if backend.requires_one_fixed_rank:
        if len(ranks) == 1:
            return fixed_rank_launch(batch, rank=ranks[0])
        return [
            fixed_rank_launch(bucket, rank=rank)
            for rank, bucket in stable_rank_buckets(batch)
        ]

    raise AdmissionError("no_compatible_rank_execution_path")

Rank is not inferred from a display name, one arbitrary adapter, or an intermediate tensor’s allocated width. The launch snapshot binds logical rank, physical rank, factor extents, slot generation, and batch-plan generation.

0x06F Tensor-Parallel Placement Must Follow The Base Projection

A multi-GPU base model adds another compatibility axis. The LoRA branch must produce the same partitioned output layout as the base projection it augments. Replicating every adapter factor on every tensor-parallel rank is a valid design alternative, but it duplicates residency and can introduce collectives that the base path did not need.

S-LoRA instead aligns adapter-factor partitions with a Megatron-style two-layer MLP. The first base matrix is column-parallel, the second is row-parallel, and the adapter communicates rank-width intermediates before placing its output in the same coordinates as the base path.

Source: S-LoRA, Figure 4. The base and LoRA branches use matching output partitions. Low-rank intermediates are gathered or reduced, and the adapter output gather is fused with the base path’s final reduction. This is the paper’s schedule, not a universal layout for every tensor-parallel runtime.

0x06F1 Reconcile Factor Orientation Before Sharding

This article has used the common checkpoint orientation:

$$A^{std}\in\mathbb{R}^{r\times d_{in}}, \qquad B^{std}\in\mathbb{R}^{d_{out}\times r},$$ $$D=X(A^{std})^\top(B^{std})^\top.$$

S-LoRA writes the same row-major operator as $XAB$, with:

$$A=(A^{std})^\top\in\mathbb{R}^{d_{in}\times r}, \qquad B=(B^{std})^\top\in\mathbb{R}^{r\times d_{out}}.$$

Nothing has changed mathematically. The names have moved with the transpose. This reconciliation must happen before interpreting a paper figure or slicing a checkpoint. Otherwise a locally valid shard can represent the wrong global operator.

0x06F2 Derive The First, Column-Parallel Projection

Let the first base matrix be:

$$W_1\in\mathbb{R}^{h\times d}, \qquad W_{1,j}\in\mathbb{R}^{h\times d/N}.$$

Its adapter factors are $A_1\in\mathbb{R}^{h\times r}$ and $B_1\in\mathbb{R}^{r\times d}$. S-LoRA splits the rank axis of $A_1$ and the output axis of $B_1$:

$$A_{1,j}=A_1[:,jr/N:(j+1)r/N] \in\mathbb{R}^{h\times r/N},$$ $$B_{1,j}=B_1[:,jd/N:(j+1)d/N] \in\mathbb{R}^{r\times d/N}.$$

Each rank first owns only a slice of the bottleneck:

$$Z_{1,j}=XA_{1,j}\in\mathbb{R}^{B\times r/N}.$$

An all-gather concatenates those slices on the rank axis:

$$Z_1=\operatorname{AllGather}_{r}(Z_{1,0},\ldots,Z_{1,N-1}) =XA_1\in\mathbb{R}^{B\times r}.$$

Every device can then produce its output shard:

$$D_{1,j}=Z_1B_{1,j}\in\mathbb{R}^{B\times d/N}.$$

The proof is a block-matrix identity:

$$\operatorname{Concat}_{d}(D_{1,0},\ldots,D_{1,N-1}) =XA_1 \begin{bmatrix} B_{1,0}&\cdots&B_{1,N-1} \end{bmatrix} =XA_1B_1.$$

The gather is not optional. Without it, a rank either has a shape mismatch or must silently use only matching diagonal rank blocks, dropping cross-shard terms.

0x06F3 Derive The Second, Row-Parallel Projection

The first projection leaves its activation sharded:

$$H= \begin{bmatrix} H_0&\cdots&H_{N-1} \end{bmatrix}, \qquad H_j\in\mathbb{R}^{B\times d/N}.$$

For $W_2\in\mathbb{R}^{d\times h}$, the base rank owns $W_{2,j}\in\mathbb{R}^{d/N\times h}$. Its adapter uses:

$$A_{2,j}\in\mathbb{R}^{d/N\times r}, \qquad B_{2,j}\in\mathbb{R}^{r\times h/N}.$$

The local shrink result is a partial sum, not a shard of the final bottleneck:

$$P_{2,j}=H_jA_{2,j}\in\mathbb{R}^{B\times r}.$$

Therefore the collective is addition:

$$Z_2=\operatorname{AllReduce}_{+}(P_{2,0},\ldots,P_{2,N-1}) =\sum_j H_jA_{2,j} =HA_2.$$

Each rank expands into one output slice:

$$D_{2,j}=Z_2B_{2,j}\in\mathbb{R}^{B\times h/N}.$$

Let $E_j(D_{2,j})$ insert that slice into columns $[jh/N,(j+1)h/N)$ of a zero $B\times h$ tensor. The base path already needs a full-width all-reduce over its partial outputs $Y^{base}_j=H_jW_{2,j}$. Add the disjoint adapter slice before that collective:

$$Y = \operatorname{AllReduce}_{+} \left( Y^{base}_j+E_j(D_{2,j}) \right) =HW_2+HA_2B_2.$$

The final reduction sums overlapping base partials while acting like an all-gather for the disjoint adapter slices. This is the fusion meant by Figure 4; it does not mean the earlier rank-width all-reduce disappeared.

0x06F4 Exact Two-Rank Tensor Trace

The Loop 6 fixture sets $B=3$ token rows, $h=d=4$, $r=N=2$, and FP16 payloads. For the first projection, the local rank slices are:

$$Z_{1,0}= \begin{bmatrix}0\\4\\4\end{bmatrix}, \qquad Z_{1,1}= \begin{bmatrix}3\\2\\2\end{bmatrix}.$$

Gathering along the bottleneck axis gives:

$$Z_1= \begin{bmatrix} 0&3\\ 4&2\\ 4&2 \end{bmatrix}.$$

The two local output shards concatenate to the unsharded reference:

$$\begin{bmatrix}D_{1,0}&D_{1,1}\end{bmatrix} = \begin{bmatrix} 6&3&0&9\\ 8&2&8&2\\ 8&2&8&2 \end{bmatrix} =XA_1B_1.$$

For the second projection, the two local bottleneck partials are:

$$P_{2,0}= \begin{bmatrix} 5&2\\-2&-1\\2&0 \end{bmatrix}, \qquad P_{2,1}= \begin{bmatrix} -4&11\\-1&4\\2&-3 \end{bmatrix}.$$

Their sum is:

$$Z_2= \begin{bmatrix} 1&13\\-3&3\\4&-3 \end{bmatrix}.$$

After the two output shards are inserted into disjoint columns, the adapter result is:

$$D_2= \begin{bmatrix} 1&15&39&25\\ -3&-3&9&9\\ 4&5&-9&-10 \end{bmatrix}.$$

The same final all-reduce combines the base partials and those disjoint slices:

$$Y= \begin{bmatrix} 2&16&41&27\\ -1&-2&10&11\\ 4&6&-6&-9 \end{bmatrix} =Y^{base}+D_2.$$

The fixture proves equality using exact integer arithmetic. It does not execute NCCL, measure overlap, or establish that this schedule is best on current hardware.

0x06F5 Weight Residency And Communication Are Separate Ledgers

Both projection adapters are fully sharded in the S-LoRA paper schedule:

Object on rank $j$	Local shape	Replicated?
$A_{1,j}$	$[h,r/N]$	no
$B_{1,j}$	$[r,d/N]$	no
$A_{2,j}$	$[d/N,r]$	no
$B_{2,j}$	$[r,h/N]$	no

For the exact FP16 fixture, the two adapters occupy 32 bytes per rank and 64 bytes across the two-rank group. Replicating all four global factors would occupy 64 bytes per rank and 128 bytes across the group. Cluster bytes and per-rank resident bytes answer different questions; neither includes allocator metadata or temporary collective buffers.

For a ring-style logical traffic ledger, one rank receives $(N-1)$ pieces of a $B\times r/N$ all-gather:

$$C_{gather}=\frac{(N-1)Br}{N} \quad\text{elements per rank}.$$

A reduce-scatter followed by all-gather moves:

$$C_{reduce}=\frac{2(N-1)Br}{N} \quad\text{elements per rank}.$$

Three Q/K/V gathers plus one output-projection reduction yield the S-LoRA paper’s additional traffic:

$$C_{LoRA} =3C_{gather}+C_{reduce} =\frac{5(N-1)Br}{N}.$$

The existing base output all-reduce moves:

$$C_{base}=\frac{2(N-1)Bh}{N}.$$

Hence:

$$\frac{C_{LoRA}}{C_{base}}=\frac{5r}{2h}$$

For $h=4096$ and $r=16$, the ratio is about $0.98\%$. In the deliberately small fixture, $h=4$ and $r=2$, so the added 30 FP16 payload bytes per rank are $1.25\times$ the existing 24-byte base payload. The toy case validates the formula; it is not chosen to make the ratio look favorable.

These $C$ terms count logical elements per rank. Multiply by bytes per element for payload bytes. They omit protocol headers, link topology, collective algorithm, startup, synchronization, temporary buffers, and overlap. The source schedule has four additional collective calls before any Q/K/V fusion: three all-gathers and one rank-width all-reduce. Small payload does not imply small latency when startup dominates. M4 owns the general collective performance model.

0x06F6 The Paper Schedule And Archived Code Are Different Evidence

S-LoRA Section 6 and Figure 4 specify the tensor-parallel design above. The final archived repository does not provide a revision-pinned executable confirmation of that design: LoraTpPartAdapter.__init__ contains assert world_size == 1. The loader still contains hidden-axis slicing code, but the guard prevents treating that code snapshot as a working multi-rank adapter path.

This distinction changes the claim:

externally verified paper claim:
    S-LoRA proposes the TP partition and communication schedule above

official-code observation at archived revision c1ddf48:
    the adapter constructor rejects world_size != 1

not established here:
    that the archived code executes Figure 4 end to end
    that the schedule is optimal on another engine or topology

The runtime manifest must therefore bind the implementation actually deployed, not merely the paper whose schedule inspired it:

global adapter contract:
    module key and base execution ABI digest
    checkpoint factor orientation and semantic scale
    global A/B shapes and logical rank

physical TP contract:
    TP degree and rank ID
    each factor's shard axis, interval, and local shape
    divisibility, uneven-shard, or padding rule
    collective kind, tensor axis, reduction operation, and order
    which adapter operation is fused into which base collective
    storage dtype, collective dtype, accumulation dtype
    implementation revision and local-kernel ABI digest

A shape-correct adapter can still be placement-wrong. Swapping gather order, concatenating partial sums, reducing an output shard that should remain sharded, or pairing a factor with another TP degree can all produce plausible local tensors while changing the global update.

Fault	Why local shapes can survive	Global failure
omit first rank-axis all-gather	slice both factors on rank axis	cross-rank bottleneck terms disappear
concatenate second partials	concatenation is shape-valid after padding	$HA_2$ is replaced by unrelated coordinates
reduce with MAX or average	collective shape is unchanged	adapter magnitude is wrong
use stale shard manifest	local extents may match	rows or columns belong to another TP degree/version
publish one rank early	each rank has valid local pointers	one request observes a mixed adapter generation

The left half follows one adapter through rank-axis gather, bottleneck reduction, and base-collective fusion. The right half separates collection-shared bases from adapter-specific cores and makes the basis/core generation an explicit runtime identity.

0x06G Compress The Adapter Collection, Not Each Adapter Independently

M6.3 and M6.4 treat each adapter as an independent low-rank update. High-cardinality serving creates another option: change the representation of the collection. This is an offline approximation problem followed by a new runtime operator and artifact contract. It is not ordinary LoRA training, and it does not make each adapter independently portable.

Source: Compress then Serve, final ICML 2025 paper, Figure 5. The paper measures memory over all 96 Mistral LoRA modules, while transfer and forward timing use one module. Joint representations reduce collection memory and transfer time in this experiment; the forward-time panel does not show a corresponding single-module latency reduction. These are source-reported results, not a benchmark reproduced by this article.

0x06G1 One SVD Cannot Solve A Shared-Basis Collection

For $n$ adapters with updates:

$$\Delta W_i=B_iA_i\in\mathbb{R}^{d_{out}\times d_{in}},$$

Compress then Serve jointly approximates them as:

$$\Delta W_i\approx U\Sigma_iV^\top$$

with shared bases:

U:       [d_out,R]       shared by a collection or cluster
V:       [d_in,R]        shared by a collection or cluster
Sigma_i: [R,R] or [R]    adapter-specific core

The collection objective is:

$$\min_{U,V,\{\Sigma_i\}} \sum_{i=1}^{n} \left\|B_iA_i-U\Sigma_iV^\top\right\|_F^2$$

This is not one SVD of one matrix. $U$ and $V$ must be shared across all $n$ terms, while every adapter keeps a different core. The paper studies two important constrained families:

Layout	Per-adapter state	Shared state	Main tradeoff
JD-Full	$R^2$	$R(d_{in}+d_{out})$	more expressive adapter-specific transform
JD-Diag	$R$	$R(d_{in}+d_{out})$	much smaller adapter payload, stronger shared-basis constraint

In JD-Full, $U^\top U=V^\top V=I_R$ and each $\Sigma_i$ is unconstrained. For fixed orthonormal $U,V$, expand one term:

$$\left\|\Delta_i-U\Sigma_iV^\top\right\|_F^2 = \|\Delta_i\|_F^2 -\left\|U^\top\Delta_iV\right\|_F^2 +\left\|\Sigma_i-U^\top\Delta_iV\right\|_F^2.$$

The last term is nonnegative, so the optimal core is:

$$\Sigma_i^*=U^\top\Delta_iV.$$

Substitution turns basis fitting into captured-energy maximization:

$$\max_{U^\top U=V^\top V=I_R} \sum_i\left\|U^\top\Delta_iV\right\|_F^2.$$

With $V$ fixed, define:

$$M(V)=\sum_i\Delta_iVV^\top\Delta_i^\top.$$

The best $U$ contains the top $R$ eigenvectors of this positive-semidefinite matrix. With $U$ fixed:

$$N(U)=\sum_i\Delta_i^\top UU^\top\Delta_i,$$

and $V$ takes its top $R$ eigenvectors. Alternating these two updates decreases the reconstruction objective. Parenthesizing through $B_iA_i$ avoids requiring one persistent $d_{out}\times d_{in}$ matrix per adapter.

JD-Diag instead sets:

$$\Sigma_i=\operatorname{diag}(s_i), \qquad s_i\in\mathbb{R}^{R},$$

and leaves $U,V$ unconstrained. Then:

$$U\operatorname{diag}(s_i)V^\top = \sum_{j=1}^{R}s_{i,j}u_jv_j^\top.$$

Every adapter is restricted to the span of $R$ shared rank-one atoms. JD-Full can use all $R^2$ cross atoms $u_pv_q^\top$. The diagonal form saves state by changing the approximation family, not by losslessly repacking a full core. The paper solves JD-Diag by coordinate descent over $U$, $V$, and the diagonal cores.

0x06G2 Exact Full-Core Reconstruction And A Diagonal Counterexample

The Loop 6 fixture uses:

$$U=V= \begin{bmatrix} 1&0\\0&1\\0&0 \end{bmatrix}$$

and three full cores:

$$\Sigma_a= \begin{bmatrix}1&0\\0&2\end{bmatrix}, \quad \Sigma_b= \begin{bmatrix}0&3\\4&0\end{bmatrix}, \quad \Sigma_c= \begin{bmatrix}2&-1\\1&1\end{bmatrix}.$$

Each update is exactly $\Delta_i=U\Sigma_iV^\top$, so JD-Full has zero squared Frobenius reconstruction error. The vectorized updates are linearly independent:

$$\operatorname{rank} \begin{bmatrix} \operatorname{vec}(\Delta_a)& \operatorname{vec}(\Delta_b)& \operatorname{vec}(\Delta_c) \end{bmatrix} =3.$$

A rank-2 JD-Diag collection lies in the span of only two shared rank-one atoms. It therefore cannot reconstruct these three independent updates exactly for any choice of shared $U,V$. Keeping the displayed bases and dropping the off-diagonal core entries gives a concrete squared residual of 27. That number is a fixture witness, not the globally optimal JD-Diag error; the span-rank argument is what proves nonzero optimal error.

The paper’s broader result is similarly conditional: JD-Full is lossless only when $R$ covers the joint column and row spaces of the collection. With noisy, independently trained adapters, that required rank can approach the sum of individual ranks. Joint decomposition is generally lossy at useful compression ranks.

0x06G3 Derive Parameter Break-Even, Not Just Compression Ratios

Let $S=d_{in}+d_{out}$ and let each source adapter have rank $r_0$. Ordinary LoRA requires:

$$P_{LoRA}=nr_0S.$$

The joint representations require:

$$P_{JD\text{-}Full}=RS+nR^2,$$ $$P_{JD\text{-}Diag}=RS+nR.$$

JD-Full saves parameters only if:

$$n(r_0S-R^2)>RS.$$

When $r_0S-R^2>0$, the break-even collection size is:

$$n>\frac{RS}{r_0S-R^2}.$$

If the denominator is non-positive, no collection size can make JD-Full smaller under this count. Similarly, JD-Diag requires:

$$n>\frac{RS}{r_0S-R}$$

when $r_0S-R>0$.

For the exact fixture, $S=6$, $r_0=R=2$, and $n=3$:

ordinary LoRA: 36 parameters / 72 FP16 bytes
JD-Full:       24 parameters / 48 FP16 bytes
JD-Diag:       18 parameters / 36 FP16 bytes
minimum integer n that saves parameters: 2 for either representation

For a larger illustrative ledger with $d_{in}=d_{out}=4096$, $r_0=16$, $R=32$, and $n=1000$:

ordinary LoRA:  131,072,000 parameters
JD-Full:          1,286,144 parameters
JD-Diag:            294,144 parameters

These are representation counts, not a promised compression ratio at fixed quality. The required shared rank $R$, decomposition error, adapter diversity, metadata, dtype, and quantization determine whether the representation is acceptable.

The ledger is for one adapted matrix shape. A full-model count must sum the shared bases and adapter cores over every adapted module and layer.

For $k$ equal-rank clusters, the payload count is:

$$P_{clustered\text{-}Full,payload} =kRS+nR^2.$$

The final paper’s GPU ledger adds one cluster assignment per adapter:

$$P_{clustered\text{-}Full,paper} =kRS+n(R^2+1).$$

The assignment is metadata, not a floating-point weight in every possible implementation. Keeping both ledgers prevents a compact paper formula from silently becoming a universal memory-layout claim.

0x06G4 The Runtime Operator Has Three Ownership Stages

For row-major token state $X$:

$$X\Delta W_i^\top \approx (XV)\Sigma_i^\top U^\top$$

Shape trace:

ordinary LoRA:
    X:[N,d_in] -> X A_i^T:[N,r] -> output:[N,d_out]

joint decomposition:
    X:[N,d_in] -> X V:[N,R]
                 -> core Sigma_i:[N,R]
                 -> shared U^T:[N,d_out]

For the fixture input:

$$X= \begin{bmatrix} 1&2&3\\0&-1&2 \end{bmatrix}, \qquad XV= \begin{bmatrix} 1&2\\0&-1 \end{bmatrix}.$$

Adapter $b$ then produces:

$$(XV)\Sigma_b^\top U^\top = \begin{bmatrix} 6&4&0\\-3&0&0 \end{bmatrix} =X\Delta_b^\top.$$

Rows sharing one collection or cluster can use one ordinary projection shape for $XV$. Their cores remain adapter-specific. JD-Diag makes the middle stage an elementwise multiply; JD-Full needs a small $R\times R$ grouped operation. The final shared $U^\top$ stage again has one basis shape per cluster.

The path has three algebraic stages instead of ordinary LoRA’s two, and $R$ can exceed $r_0$. Lower resident bytes therefore do not automatically imply lower TPOT. The final paper explicitly reports large memory and transfer reductions without a corresponding one-module forward-latency reduction in its timing experiment.

Tensor parallelism and collection compression are not independent. The shared $U,V$ bases need the same physical shard contract as the projection they replace, while each $\Sigma_i$ core must be paired with that exact basis generation. A basis-shared batch can still require cluster grouping before its TP collectives.

0x06G5 Offline Conversion Is A Training And Evaluation Boundary

The paper compresses a set of already trained LoRAs. The decomposition has access to adapter weights; it does not jointly fine-tune the base model. Its experiments normalize every $B_iA_i$ to unit Frobenius norm before fitting and restore the original norms for reconstruction and evaluation. They limit the main alternating runs to ten iterations rather than claiming full convergence.

An operational conversion pipeline is:

def build_collection_bundle(source_adapters, base_abi, module_key, config):
    frozen = freeze_ordered_source_set(source_adapters)
    for adapter in frozen:
        validate_base_module_orientation_scale(adapter, base_abi, module_key)

    deltas, norms = [], []
    for adapter in frozen:
        delta = adapter.semantic_scale * adapter.B @ adapter.A
        norm = frobenius_norm(delta)
        require(norm > 0)
        deltas.append(delta / norm)
        norms.append(norm)

    if config.method == "JD-Full":
        U, V, cores = alternating_top_eigenspaces(
            deltas,
            rank=config.shared_rank,
            max_iterations=config.max_iterations,
        )
    elif config.method == "JD-Diag":
        U, V, cores = coordinate_descent_diagonal_cores(deltas, config)
    else:
        raise UnsupportedRepresentation(config.method)

    restored_cores = [norm * core for norm, core in zip(norms, cores)]
    metrics = evaluate_candidate(
        frozen,
        U,
        V,
        restored_cores,
        reconstruction_manifest=config.reconstruction_suite,
        task_manifest=config.held_out_task_suite,
    )
    require_release_thresholds(metrics, config.release_policy)

    return emit_immutable_collection_bundle(
        source_set=frozen,
        base_abi=base_abi,
        module_key=module_key,
        bases=(U, V),
        cores=restored_cores,
        metrics=metrics,
    )

Frobenius reconstruction error is a proxy, not a substitute for downstream evaluation. The paper reports a nonlinear relation between reconstruction and task performance. Rank, cluster count, iteration count, normalization, and release thresholds therefore belong to an immutable experiment manifest.

0x06G6 Clustering Trades Shared Residency For A Smaller Local Subspace

If one basis cannot approximate a diverse collection, adapters can be clustered. Cluster $c$ owns $U_c,V_c$ while each adapter owns $\Sigma_i$ and a cluster assignment. That changes scheduler and artifact state:

The paper initializes clustering by fitting one joint basis and running k-means on the resulting cores. It then alternates two operations:

fit step:
    refit one JD model independently inside each current cluster

assignment step:
    assign each adapter to the cluster with minimum reconstruction error

stop:
    no assignment changes, or a declared iteration/validation limit is reached

More clusters increase shared-basis residency and can fragment runtime batches, but can reduce reconstruction error at a fixed per-cluster rank. The paper reports JD-Full without clustering as effective up to roughly 100 adapters in its evaluated collection and uses clustering at larger scales. That is a source-reported recommendation for its tasks and hardware, not a universal threshold.

0x06G7 A Core Is Meaningless Without Collection Identity

adapter identity:
    base revision
    ordered source adapter set and source digests
    module key, orientation, semantic scale, and base execution ABI
    decomposition method and rank R
    optimizer, normalization, rank, cluster count, and evaluation manifest
    collection and basis version
    cluster ID
    Sigma_i revision
    basis and core digests

scheduler affinity:
    prefer rows sharing a resident basis/cluster
    but do not violate queueing and SLO limits to maximize affinity

Adding, removing, or retraining one adapter can require reclustering or recomputing shared bases. A compressed core is therefore not independently meaningful. Loading $\Sigma_i$ with the wrong $U,V$ version is as incorrect as loading LoRA factors against the wrong base model.

Publication must be generation-atomic:

freeze source set
-> validate base/module/orientation/scale
-> fit bases and cores offline
-> evaluate reconstruction and held-out task metrics
-> emit immutable bundle
-> load complete bundle into a staging generation
-> validate basis/core/kernel digests on every TP rank
-> publish one collection generation atomically
-> retire the old generation after its last reference

Publishing a new basis before all matching cores are ready creates a mixed operator even when every tensor has the expected shape.

0x06G8 Method Alternatives And Paper Contracts

Representation	Shared state	Per-adapter state	Runtime middle stage	Main limit
original LoRA	none	$r_0S$	two adapter-specific projections	residency and transfer scale linearly with $n$
per-adapter SVD	none	reduced-rank factors per adapter	two adapter-specific projections	cannot amortize bases across the collection
JD-Diag	$RS$	$R$	elementwise core	only $R$ shared rank-one atoms
JD-Full	$RS$	$R^2$	small grouped matrix multiply	larger cores; useful rank can grow with diversity
clustered JD-Full	$kRS$	$R^2$ plus assignment	group by cluster, then by adapter core	more shared residency and scheduling fragmentation
keep original and page	no approximation	original factors	existing Punica/S-LoRA/vLLM path	load/offload pressure remains

The final paper uses custom vLLM changes and Punica-style grouped operations, and publishes pseudocode for an A -> Sigma -> B path. No revision-pinned official implementation is used as evidence here: the authors’ public code link resolves to the Lots-of-LoRAs model/data organization, not a source tree for the modified serving runtime. Paper pseudocode and measured results remain paper evidence.

Source-reported results include experiments with up to 1000 trained LoRAs and a serving sweep to 1024 unique adapters. The paper reports about $1.6\times$ throughput improvement over its matched-memory vLLM multi-LoRA baseline at the largest setting and about 80% of the no-LoRA/single-merged-model throughput. Those numbers depend on the selected rank/cluster setting, quality threshold, request trace, model, implementation, and hardware. They are not a prediction for this blog’s toy fixture or another runtime.

0x06G9 Collection Compression Failure Lab

Failure	Detection	Required response
wrong source factor orientation or scale	reconstruct source $B_iA_i$ under manifest before fitting	reject conversion
one core paired with stale basis	basis/core digest or generation mismatch	reject admission/launch
source set changes under same collection version	ordered source-set digest differs	create a new collection version
parameter savings claimed with non-positive break-even denominator	evaluate symbolic inequality	report no count-model break-even
JD-Diag assumed as expressive as JD-Full	compare collection span and validation error	raise rank, use full cores, or cluster
low reconstruction error treated as quality proof	held-out task manifest absent	block release
lower bytes treated as lower TPOT	runtime measurement absent	keep latency claim unverified
one TP rank publishes new basis early	distributed generation snapshot differs	fail closed and roll back staging generation

0x06H Executable Loop 6 Oracle

The repository fixture closes the bounded algebraic and metadata contract:

two-rank first projection:
    shard A on rank axis and B on output axis
    all-gather bottleneck slices
    prove concatenated output equals X A B

two-rank second projection:
    shard input/A on input axis and B on output axis
    all-reduce bottleneck partials
    fuse disjoint adapter output slices into the base final all-reduce

three-adapter collection:
    prove JD-Full reconstruction and runtime output exactly
    prove collection span rank 3 exceeds rank-2 JD-Diag capacity
    derive parameter counts and integer break-even points
    bind source adapters, bases, cores, TP ABI, and generations

Fifty-eight negative fixtures reject source overclaims, wrong axes, missing or reordered collective contracts, malformed shards, non-orthonormal full bases, stale source/core/basis digests, mixed collection generations, and empirical claims outside the oracle’s evidence surface.

0x07 Merge, Unmerge, And Hotswap Are Different Operations

The word merge is overloaded. A framework can add one delta into one layer, remove adapter wrappers from an entire model, replace tensors in an allocated adapter slot, or publish a new serving generation. Those operations do not have the same rollback, concurrency, or identity contract.

0x07A Operation Taxonomy And Object Identity

Operation	Weight state after the operation	Adapter factors retained?	Request-time selection?	Canonical rollback
live adapter	immutable base plus selected factors	yes	yes	stop selecting the adapter
merge_adapter	delta added to wrapped base layers	yes	no for merged adapter	arithmetic unmerge, with precision caveat
arithmetic unmerge	stored delta subtracted from current base tensors	yes	restored	reload clean base when exactness matters
merge_and_unload	merged plain-model modules	no LoRA wrappers	no	reload immutable base artifact
unload	plain-model modules without adding delta	wrappers removed	no	reload adapter to restore adaptation
framework hotswap	factor tensors replaced inside one adapter name/slot	yes	adapter remains live	load old factors again
runtime generation swap	catalog publishes a new immutable slot handle	old generation retires later	yes	publish another generation
immutable merged release	new full checkpoint or deployment artifact	not required at runtime	no	release/traffic rollback

The identities differ as well. A live adapter revision is not a merged base revision, and a mutable resident slot is not the semantic adapter artifact:

$$\Delta_a = s_a B_a A_a$$ $$M_{\text{merged}} = (base\_revision, ordered\_adapter\_set, dtype\_schedule, quantizer, target\_signature)$$ $$H = (slot\_id, slot\_generation, pointer\_generation, artifact\_digest)$$

Changing any field creates a different executable object even if a human-facing alias remains tenant-a.

0x07B Derive Live And Merged Equivalence

Let requests be rows of $X$, use the checkpoint orientation $W\in\mathbb{R}^{d_{out}\times d_{in}}$, and let $A\in\mathbb{R}^{r\times d_{in}}$ and $B\in\mathbb{R}^{d_{out}\times r}$. The live branch is

$$Y_{\text{live}} = XW^\top + s(XA^\top)B^\top.$$

Associativity gives

$$s(XA^\top)B^\top = X(sBA)^\top = X\Delta_a^\top.$$

Therefore a merged weight

$$W_a' = W + \Delta_a$$

produces

$$Y_{\text{merged}} = X(W_a')^\top = XW^\top + X\Delta_a^\top = Y_{\text{live}}.$$

This is an algebraic identity over one consistent arithmetic and tensor orientation. It does not prove equality after independently rounding the live and merged paths, dequantizing a base, changing accumulation dtype, omitting a target module, or applying a different scale convention.

0x07B1 Exact Old And New Adapter Trace

The Loop 7 fixture uses

$$W= \begin{bmatrix} 2 & -1 & 0\\ 1 & 3 & -2 \end{bmatrix}.$$

The old rank-two adapter has $s_o=1/2$ and produces

$$\Delta_o = \begin{bmatrix} 1/2 & 2 & 1/2\\ -1/2 & 1 & 1 \end{bmatrix}.$$

For $x_o=[1,2,-1]$, both the live and merged operators return

$$y_o=[4,19/2].$$

The new adapter has logical rank one but occupies a physical rank-two slot. Its second row of $A_n$ and second column of $B_n$ are zero, so padding cannot inherit bytes from the old adapter. With $s_n=1/4$:

$$\Delta_n = \begin{bmatrix} 1/2 & -1/4 & 0\\ 3/2 & -3/4 & 0 \end{bmatrix}, \qquad x_n=[2,0,1], \qquad y_n=[5,3].$$

The operator shape remains rank two; the semantic work remains rank one. This is the same logical-rank versus physical-capacity distinction established in Loop 4, now applied to slot replacement.

0x07C Arithmetic Unmerge Is Not Exact Rollback

In exact arithmetic, subtracting the same delta recovers the base:

$$W = (W + \Delta_a) - \Delta_a.$$

A stored tensor does not execute exact real arithmetic. Let $Q$ denote one rounding or quantization step. A representative merge/unmerge path is

$$W_m = Q(Q(W)+Q(\Delta_a)),$$ $$W_u = Q(W_m-Q(\Delta_a)), \qquad E_u = W_u-W.$$

There is no identity requiring $E_u=0$. A binary16 scalar counterexample is enough:

clean base:                  2048
delta:                          1
binary16 merge:              2048 + 1 -> 2048
binary16 arithmetic unmerge: 2048 - 1 -> 2047
all values finite:           yes
exact round trip:            no

The unit in the last place at 2048 is 2. The merge loses the increment, while the subsequent subtraction rounds to the adjacent representable value. A finiteness check accepts this result because neither NaN nor infinity appears. Quantized bases add dequantize, requantize, scale, zero-point, packing, and saturation decisions, so an exact arithmetic rollback claim is even weaker.

The operational rule is:

immutable clean base is the rollback source
merged checkpoint is a new base revision
arithmetic unmerge is a convenience operation, not artifact recovery

0x07D What Pinned PEFT Actually Guarantees

The PEFT v0.19.1 source establishes narrower contracts than a generic claim that “safe merge is transactional”:

1. Linear.merge(safe_merge=True) clones one base-layer weight, adds that layer’s delta, checks that the result is finite, installs the clone, and only then records the adapter as merged.
2. unmerge() pops merged adapters in reverse order and subtracts each delta. It therefore inherits finite-precision drift.
3. get_delta_weight() can temporarily cast CPU FP16/BF16 factors to FP32 for the matrix multiplication and cast the delta back. Dtype scheduling is part of the result.
4. merge_adapter retains wrappers and factors; merge_and_unload merges and replaces wrappers with base modules; unload removes wrappers without merging.
5. Whole-model merge/unload iterates modules. A later module can fail after an earlier one changed, so layer-local safe_merge is not a whole-model atomic transaction or rollback mechanism.

For compiled hotswap, the pinned preparation path can tensorize scaling and pad factor storage to a configured maximum rank. It must run after the first adapter is loaded and before compilation, and differing target layers remain unsupported. The hotswap path dry-runs incoming keys, rejects unexpected keys, zeros stale old weights for missing incoming keys, updates scaling, and then uses tensor swap for the ordinary path or in-place copy for compiled tensors. Unused physical-rank tails are zero-filled.

Those are useful framework mechanics. They still do not provide one global all-module transaction, compensating rollback after a partial copy, request leases, stream/event fencing, or production multi-request concurrency safety. Loop 7 adds those as rewrite contracts, not as PEFT claims.

0x07E vLLM Registration Is Not Same-Alias Hotswap

The pinned vLLM v0.12.0 serving surface serializes adapter loading by LoRA name, rejects a duplicate name, allocates a new integer ID, asks the engine to add the adapter, and publishes the request in its serving registry. Unload removes the registered name. Worker-side mutation is explicitly not thread-safe and relies on the single-threaded core engine loop; activation copies LoRA tensors into a free GPU slot.

Its LoRARequest carries name, integer ID, and path, but no slot generation or pointer generation. Therefore the stricter same-alias, overlapping-generation protocol below is not a description of vLLM v0.12.0. It is the runtime contract required if an implementation chooses to offer that feature.

0x07F A Hotswap Is A Publication Transaction

A slot handle is valid only as one tuple:

$$H_g=(slot\_id,slot\_generation,pointer\_generation,artifact\_digest).$$

The catalog maps an alias at generation $g$ to that immutable handle:

$$C_g(alias)\rightarrow H_g.$$

The exact fixture starts with

catalog generation 8
tenant-a -> slot 7 / slot generation 12 / pointer generation 31 / old digest

old request lease
request-old -> the same four-part old handle
refcount(slot 7) = 1
last-use event(slot 7) = incomplete

The replacement is staged in free slot 8. It is not visible merely because the copy began or the factor shapes fit. Publication requires

$$copy\_event.complete \land digest\_valid \land layout\_valid \land padded\_tail=0.$$

After one atomic catalog update:

catalog generation 9
tenant-a -> slot 8 / slot generation 21 / pointer generation 41 / new digest

request-old keeps the old handle and output [4, 19/2]
request-new binds the new handle and output [5, 3]
slot 7 becomes retiring; slot 8 becomes active

0x07G Shadow-Copy Publication Algorithm

def publish_replacement(alias, artifact):
    old = catalog.resolve(alias)  # immutable generation handle
    validate_artifact(artifact, base_abi, target_signature)

    shadow = reserve_free_slot(next_generation=True)
    try:
        zero_physical_rank_tail(shadow, artifact.logical_rank)
        copy_all_module_tensors(shadow, artifact)
        validate_staged_digest_layout_and_finiteness(shadow, artifact)

        copy_done = record_event(copy_stream)
        publication_stream.wait_event(copy_done)

        new = shadow.freeze_handle(
            artifact_digest=artifact.digest,
            pointer_generation=shadow.pointer_generation + 1,
        )
        catalog.compare_and_publish(alias, expected=old, replacement=new)
        mark_active(new)
        mark_retiring(old)
        return new
    except Exception:
        # Catalog still points at old. No new request can bind shadow.
        discard_shadow_and_bump_generations(shadow)
        raise

The event semantics are deliberately modest. CUDA documents that a recorded event completes after preceding work in its stream and that a waiting stream’s subsequent work waits for that event. This article does not execute a CUDA event or prove a production control plane. The fixture checks the required state ordering only.

0x07H Request Leases Prevent Mixed Generations

A request resolves the alias once at admission and retains the resulting lease for its lifetime:

def bind_request(request, alias):
    handle = catalog.resolve(alias)
    reject_if_stale(handle)
    increment_refcount(handle)
    request.adapter_lease = handle

def build_launch(request):
    handle = request.adapter_lease  # do not re-resolve alias per iteration
    assert pointer_table.matches(handle)
    return launch_snapshot(handle)

Re-resolving tenant-a during every decode iteration can move one request from old factors to new factors after publication. That would also violate the KV provenance contract in 0x08 whenever the adapter changes K/V-producing projections. A stable request lease closes both the factor and cache identity.

The old slot may be reclaimed only when

$$refcount(H_{old})=0 \land last\_use\_event(H_{old}).complete.$$

The reference count protects submitted and queued logical users. The event protects asynchronous device work after host ownership was released. Neither substitutes for the other. Reclamation increments both slot and pointer generations, so a delayed stale handle cannot become valid again when slot 7 is reused: this is the ABA defense.

0x07I Failure, Rollback, And Post-Publish Recovery

The fixture injects failure while copying the new module tensors, before publication:

catalog remains generation 8 and still names old slot 7
old request output remains [4, 19/2]
no request can bind the shadow slot
partial new weights are never visible
shadow slot returns to free with bumped slot/pointer generations

This is rollback because the visible catalog never changed. Once generation 9 is published and requests bind it, silently mapping the alias back to generation 8 is not an in-place undo: it can strand new requests, violate refcounts, and reintroduce an already-retiring handle. Post-publication recovery must stage and publish another generation, with the old artifact bytes or another approved revision, under the same copy and retirement rules.

0x07J Cost Ledger And Design Alternatives

Shadow publication pays overlap memory:

$$M_{peak}=M_{old}+M_{new}+M_{staging\ metadata}.$$

It also pays copy and validation time before publication, plus old-generation retirement time afterward. Those costs are not adapter-kernel TPOT. They belong to control-plane availability and cold-transition ledgers.

Design	Extra resident capacity	Availability during change	Required fence	Appropriate use
drain then overwrite in place	no extra slot	blocks new work and waits old work	old refcount zero and last-use event complete	memory-constrained, infrequent maintenance
shadow copy then publish	one compatible slot	old and new generations overlap	copy event before publish; retire event before reclaim	same alias with low interruption
load under a new alias	one slot	caller migrates explicitly	normal request binding and old-alias drain	simplest versioned API
immutable merged release	full model artifact/deployment	separate rollout	release validation and traffic cutover	stable single-adapter product

PEFT compiled hotswap optimizes replacement inside a prepared model; vLLM’s pinned API loads a distinct name/ID; the shadow protocol specifies a stronger serving feature. These contracts solve different problems and should not be presented as interchangeable implementations.

0x07K Lifecycle Failure Lab

Failure	Why shape checks miss it	Required response
arithmetic unmerge treated as exact artifact rollback	every tensor remains finite and shape-correct	reload immutable clean base
per-layer safe merge called whole-model atomic	an early layer can change before a later failure	stage a new model artifact or implement compensating transaction
compiled slot tail not zeroed after rank decrease	allocated shapes are unchanged	zero unused A rows and B columns before validation
missing incoming key leaves old factor bytes	old tensor still has valid shape	explicit zero or reject according to compatibility policy
alias published before copy event	catalog and pointer shapes look valid	wait event, validate digest/layout, then publish once
request re-resolves alias each decode iteration	alias is valid before and after update	retain admission-time generation lease
retiring slot reclaimed on refcount alone	host users are gone but GPU work can remain	require refcount zero and completed last-use event
retiring slot reclaimed on event alone	queued/submitted logical owners can remain	require event and reference conditions
reused slot keeps same generation	stale pointer can pass identity comparison	bump slot and pointer generations
pre-publication failure exposes staged pointer	factor contents can be partial across modules	keep shadow unreachable and discard it
post-publication rollback mutates catalog backward	new-generation requests already exist	publish another generation transaction
framework behavior presented as runtime proof	library code lacks request and event ownership	label source claim and rewrite contract separately

0x07L Executable Loop 7 Oracle

The local rational and integer oracle proves only the bounded contract:

old and new live outputs equal their merged outputs exactly
logical rank 1 equals zero-padded physical rank 2
binary16 arithmetic merge/unmerge drifts 2048 -> 2047
old and new requests remain bound to distinct immutable handles
shadow copy failure leaves the old catalog and output unchanged
retirement needs both refcount zero and last-use event completion
slot and pointer generations advance on aborted staging and reclaim
86 one-field failures reject source overclaims, malformed tensors,
identity drift, unsafe ordering, early reclamation, and claim inflation

It imports neither PEFT nor vLLM, copies no GPU tensor, records no CUDA event, compiles no graph, and measures no latency, throughput, or model quality.

0x08 Adapter And KV Compatibility

An adapter does not merely accompany a request. It may become an ancestor of the K/V rows already stored for that request. The owner question is therefore:

Given a fixed token interval, which adapter execution facts change the mathematical K/V rows, which facts change only later logits, and which physical runtime facts must stay outside the semantic prefix key?

The answer is not “check whether the adapter targets k_proj or v_proj.” Direct K/V targets are the simplest incompatibility, but a query, output, or MLP update in an earlier layer can change the hidden state projected into K/V by every downstream layer. Conversely, an output-head-only adapter can change prompt logits while leaving every K/V row unchanged. Exact reuse needs an architecture dependency closure and an interval-specific activation schedule, not a module-name shortcut.

This section owns the adapter projection supplied to M5’s cache contract. It does not reimplement M5.3’s block hash, radix tree, allocator, reference counts, copy-on-write, or immutable prefix-match receipt.

0x08A Derive K/V From Effective Per-Token Operators

Let $m_a(t)\in\{0,1\}$ say whether adapter $a$ is active at token position $t$. For module $u$ in layer $\ell$, define the effective operator

$$W^{(a)}_{\ell,u}(t) = W_{\ell,u}+m_a(t)\Delta W_{\ell,u,a}.$$

For the pre-normalized hidden row $x_{\ell,t}$,

$$K^{(a)}_{\ell,t} = x_{\ell,t} \left(W^K_\ell+m_a(t)\Delta W^K_{\ell,a}\right)^\top,$$ $$V^{(a)}_{\ell,t} = x_{\ell,t} \left(W^V_\ell+m_a(t)\Delta W^V_{\ell,a}\right)^\top.$$

A direct K update produces

$$K^{(a)}_{\ell,t}-K^{(\mathrm{base})}_{\ell,t} = m_a(t)x_{\ell,t}\Delta W^{K\top}_{\ell,a}.$$

That equation is necessary but not sufficient. The hidden row is itself a causal recurrence:

$$x_{\ell,t} = \operatorname{Norm}_\ell\!\left( h_{\ell,t}\right), \qquad h_{\ell+1,t} = F_\ell\!\left( h_{\ell,\le t}; W^{(a)}_{\ell,Q,K,V,O,\mathrm{MLP}}(\le t) \right).$$

Even when $\Delta W^K_{\ell,a}=\Delta W^V_{\ell,a}=0$, an adapted ancestor can make $x_{\ell,t}$ differ. Cache identity follows the full producer graph, not only the final projection names.

0x08B The Cache-Ancestor Closure

For a conventional serial decoder block, a conservative ancestor set for one cached row is

$$\mathcal C_{\ell,t}^{KV} = \left\{K_{\ell,t},V_{\ell,t}\right\} \cup \bigcup_{j<\ell}\; \bigcup_{u\le t} \left\{Q_{j,u},K_{j,u},V_{j,u},O_{j,u},\mathrm{MLP}_{j,u}\right\}.$$

The same-layer Q, O, and MLP operators are not ancestors of K/V that were already projected at that layer. They are ancestors of downstream-layer K/V, and their effects can propagate to later tokens through causal attention. A parallel attention/MLP block, post-norm model, recurrent state-space layer, or other architecture needs its own graph. The graph revision is therefore part of the compatibility manifest.

Define an affected-row predicate

$$\chi_{a,\ell,t}^{KV} = \mathbf 1\!\left[ \exists n\in\mathcal C_{\ell,t}^{KV}: m_a(\operatorname{pos}(n))\Delta W_{a,n}\ne0 \right].$$

Two K/V rows are compatible only if their exact inputs, position contract, representation, and effective operators agree throughout the ancestor closure:

$$\operatorname{CompatKV}_{\ell,t}(a,b) \Longrightarrow \left. \mathcal E_a \right|_{\mathcal C_{\ell,t}^{KV}} = \left. \mathcal E_b \right|_{\mathcal C_{\ell,t}^{KV}},$$

where $\mathcal E$ is the immutable execution manifest. Equality of raw bytes or tensor shapes does not prove this condition. The implication is written in the safe direction: an implementation may conservatively reject equivalent rows if it lacks a proof, but it may not accept incompatible rows because a target-name allowlist looked harmless.

0x08C Exact Two-Layer Dependency Witness

The local Loop 8 fixture uses a two-token, two-layer scalar-head witness. It keeps the softmax arithmetic exact by selecting logits whose exponentials are integers. The input rows are

$$x_{0,0}=[1,0], \qquad x_{0,1}=[1,1].$$

Choose

$$W^K_0= \begin{bmatrix}0\\1\end{bmatrix}, \qquad W^V_0= \begin{bmatrix}1\\2\end{bmatrix}.$$

Layer 0 therefore stores

$$K_0=[0,1], \qquad V_0=[1,3].$$

A Q-only adapter leaves those rows unchanged. For token 1, let the base query produce logits $[0,0]$ and the adapted query produce $[0,\log 3]$. Then

$$\operatorname{softmax}([0,0]) = \left[\frac12,\frac12\right],$$ $$\operatorname{softmax}([0,\log 3]) = \left[\frac14,\frac34\right].$$

With values $[1,3]$, the attention contexts are

$$c_{\mathrm{base}}=2, \qquad c_{Q\text{-adapter}}=\frac52.$$

Let the output direction be $[1,0]$. After the residual update,

$$h^{(\mathrm{base})}_{1,1}=[3,1], \qquad h^{(Q\text{-adapter})}_{1,1}=\left[\frac72,1\right].$$

Now project the next layer with

$$W^K_1= \begin{bmatrix}2\\0\end{bmatrix}, \qquad W^V_1= \begin{bmatrix}1\\1\end{bmatrix}.$$

The downstream cache rows differ:

$$K^{(\mathrm{base})}_{1,1}=6, \qquad K^{(Q\text{-adapter})}_{1,1}=7,$$ $$V^{(\mathrm{base})}_{1,1}=4, \qquad V^{(Q\text{-adapter})}_{1,1}=\frac92.$$

The trace proves the missing implication:

layer-0 Q update
    -> layer-0 K/V rows unchanged
    -> layer-0 attention output changes
    -> layer-1 hidden input changes
    -> layer-1 K/V rows change

For comparison, a direct K delta $\Delta W^K_0=[1,1]^\top$ changes layer-0 K from [0, 1] to [1, 3] immediately. Both paths invalidate ordinary raw-adapter cross-reuse, but for different graph reasons.

0x08D K/V Compatibility Is Not Prompt-Output Compatibility

Suppose an adapter changes only the final LM head. For fixed final hidden row $h=[3,1]$, use

$$W_{\mathrm{LM}}= \begin{bmatrix}1&0\\0&1\end{bmatrix}, \qquad \Delta W_{\mathrm{LM}}= \begin{bmatrix}1&0\\0&-1\end{bmatrix}.$$

The logits change from

$$z_{\mathrm{base}}=[3,1] \quad\text{to}\quad z_{\mathrm{adapter}}=[6,0],$$

while every K/V row remains unchanged. This creates two independent questions:

Reused object	Required equality
K/V prefix only	cache-producing ancestor closure
cached final hidden state	hidden-state producer closure
cached prompt logits	hidden-state closure plus LM-head operator
sampled continuation	K/V and logits for the fixed prefix, then the actual token/RNG policy

A runtime may reuse the K/V payload for an LM-head-only adapter, but it may not reuse base prompt logits as adapter logits. Likewise, equal K/V does not mean future generated token IDs will remain equal. M5 keys exact past state; the sampler owns the future branch.

0x08E Activated LoRA Makes Activation A Position Function

Activated LoRA changes ordinary LoRA’s always-on schedule. Let $t_{\mathrm{invoke}}$ be the first token of the chosen invocation occurrence:

$$m_a(t)=\mathbf 1[t\ge t_{\mathrm{invoke}}].$$

The projected matrices become piecewise:

$$K^{(a)}_{1:T} = \begin{bmatrix} X_{1:t_{\mathrm{invoke}}-1}W^K\\ X_{t_{\mathrm{invoke}}:T}(W^K+\Delta W^K_a) \end{bmatrix},$$

with corresponding equations for Q, V, and any adapted MLP/operator. The important boundary is strict:

[0, t_invoke)     base execution; eligible for base/aLoRA sharing
[t_invoke, T)     adapter execution; adapter-specific cache

For tokens=[11,12,99,100] and invocation tokens [99,100], pinned PEFT v0.19.1 derives

$$t_{\mathrm{invoke}}=2, \qquad o=T-t_{\mathrm{invoke}}=2,$$

and applies the adapter to the last o rows:

position:       0      1      2      3
token:         11     12     99     100
adapter mask: false  false   true   true
reusable base frontier: [0,2)

PEFT searches for the last exact invocation-token occurrence. For [99,100,11,99,100,12], the last start is 3, the backward offset is 3, and rows 3 through 5 are adapted. Tokenization therefore belongs to the activation contract: a matching text substring is insufficient if its token IDs do not contain the exact invocation sequence.

0x08F Why The Pre-Invocation Prefix Is Equal

The aLoRA paper’s proposition is an induction over causal positions and layer depth, not an assertion that arbitrary LoRA caches happen to be close.

1. For every $t<t_{\mathrm{invoke}}$, the adapter mask is zero for every adapted operator.
2. Layer 0 therefore receives the same token/position input and uses the same effective Q/K/V/MLP operators as the base model.
3. Causal attention at those positions cannot observe later activated rows.
4. The layer output is equal, so the next layer receives equal hidden rows.
5. Induction over layers proves all internal states, including K/V, equal on $[0,t_{\mathrm{invoke}})$.

Formally,

$$\forall \ell,\;t<t_{\mathrm{invoke}}: \quad h^{(a)}_{\ell,t}=h^{(\mathrm{base})}_{\ell,t}, \quad K^{(a)}_{\ell,t}=K^{(\mathrm{base})}_{\ell,t}, \quad V^{(a)}_{\ell,t}=V^{(\mathrm{base})}_{\ell,t}.$$

After invocation, that equality no longer licenses cross-adapter reuse. The paper explicitly treats adapter-generated suffix cache as adapter-specific. For $N$ adapters sharing $T_{cache}$ base rows and processing $T_{new}\ll T_{cache}$ activated rows, its source-reported asymptotic ledger is

$$C_{\mathrm{aLoRA}} = O(NT_{cache}T_{new}), \qquad M_{\mathrm{aLoRA,extra}} = O(NT_{new}),$$

versus ordinary LoRA’s recomputed-prefix terms

$$C_{\mathrm{LoRA}} = O\!\left(N(T_{cache}+T_{new})^2\right), \qquad M_{\mathrm{LoRA,extra}} = O\!\left(N(T_{cache}+T_{new})\right).$$

These are paper-model asymptotics, not a local latency reproduction. Block granularity, batching, kernel support, cache-copy policy, and scheduling can change realized savings.

0x08G Training And Pinned PEFT Runtime Contract

aLoRA’s cache property is created during training. The paper’s SFT contract uses an input/context followed by an invocation and completion:

context tokens:
    adapter disabled
    normally excluded from completion loss

invocation tokens and completion:
    adapter enabled
    completion labels train the specialized behavior

The authors adapt attention projections and note that MLP adaptation can use the same position-gated construction. They also report that aLoRA can require higher rank than ordinary LoRA in some tasks. Those are training design choices; cache equivalence still comes from the zero adapter mask before the boundary.

At revision ba6a190..., PEFT implements a narrower concrete contract:

Pinned behavior	Serving consequence
find last exact invocation-token subsequence	activation depends on full tokenized input, not a text search
return offset T - start and mask the last offset rows	the first invocation token is adapted
invocation absent -> None offset -> base path	a product expecting specialization should reject rather than silently accept base behavior
inputs_embeds without token IDs -> aLoRA disabled with warning	side-input-only requests need an explicit offset or fail-closed policy
post-activation cache documented as adapter-tainted	preserve a clean fork before activation if another branch needs base cache
merge/unmerge unsupported	selective per-token execution cannot be replaced by one static merged weight
beam search unsupported in the pinned path	serving admission must reject unsupported generation mode

PEFT’s DynamicCache examples show two useful patterns. Base-to-aLoRA can pass the clean base cache into the activated continuation. aLoRA-to-base or aLoRA-to-another-aLoRA preserves a deep copy/fork of the clean shared prefix before the first branch taints its suffix. Those examples demonstrate caller cache handling; they do not implement M5.3’s multi-tenant prefix index, generation receipt, or authorization boundary.

0x08H Project Adapter State Into An Interval Execution Digest

M5.1 defines the logical cache ABI. M5.3 defines the physical layout, candidate key, namespace, references, and exact-reuse receipt. M6.5 supplies one field that neither article can infer alone:

$$E_{[s,e)} = \operatorname{SHA256}\!\left( \operatorname{Canonical}( \text{effective operator identities over } \mathcal C^{KV}_{\ell,t},\ s\le t<e) \right).$$

The interval-specific cache projection is:

covered token IDs and position IDs
base execution ABI
interval execution digest E_[s,e)
dependency-graph revision
cache schema, dtype, quantization, and local shard
position / RoPE / visibility contract
side-input identity
isolation namespace

The following are ownership or future-policy fields, not mathematical K/V compatibility fields:

request ID
adapter catalog alias
resident slot ID
slot generation
pointer generation
physical K/V block IDs
sampling policy

The raw requested adapter ID is retained in the immutable request manifest, but it is not blindly copied into every interval key. For ordinary always-on LoRA, $E_{[0,e)}$ includes the adapter revision from position 0. For aLoRA, an interval wholly before invocation normalizes to the base execution digest, allowing several adapters to share it. An interval touching or following the boundary includes the active adapter revision and activation schedule.

This avoids two opposite bugs:

raw adapter ID in every pre-invocation block:
    safe but destroys aLoRA's intended base-prefix sharing

adapter omitted after activation:
    unsafe cross-adapter hit over different K/V rows

If a prefix engine indexes only full token blocks and the activation boundary falls inside a block, the reusable indexed length may stop at the preceding full block even though individual rows before the boundary are equal. Partial tail publication and lookup remain M5.3 policies.

0x08I Prefix Lookup And Publication Protocol

def acquire_adapter_compatible_prefix(request, cache_index):
    manifest = freeze_request_model_adapter_activation_manifest(request)
    execution = derive_interval_execution_digests(
        tokens=manifest.token_ids,
        positions=manifest.position_ids,
        base_abi=manifest.base_execution_abi,
        adapter=manifest.adapter_artifact,
        activation=manifest.activation_schedule,
        dependency_graph=manifest.dependency_graph_revision,
    )

    namespace = authorize_cache_namespace(manifest.isolation)
    candidate = cache_index.lookup_longest(
        namespace=namespace,
        token_ids=manifest.token_ids,
        execution_digests=execution,
        cache_abi=manifest.cache_abi,
    )

    receipt = validate_and_acquire_m5_prefix_receipt(candidate, manifest)
    branch = fork_copy_on_write(receipt)

    first_incompatible = receipt.covered_end
    suffix = run_prefill_suffix(
        branch,
        manifest.token_ids[first_incompatible:],
        activation=manifest.activation_schedule,
    )
    return publish_complete_suffix_rows(branch, suffix)

The ordering is mandatory:

freeze semantics
    -> derive interval execution
    -> authorize namespace
    -> lookup and validate candidate
    -> acquire immutable M5 receipt
    -> fork shared prefix
    -> compute first incompatible suffix
    -> publish all-layer suffix rows

A cache mismatch rejects the candidate without mutating the producer. Failure before suffix publication discards unpublished rows and retains the old receipt. Failure after a successor publication is repaired by another generation transaction. Old adapter rows are never relabeled as new adapter or base rows.

0x08J Mid-Request Switch Cases

Switch	Reusable K/V in the four-token fixture	Required action
ordinary LoRA A -> ordinary LoRA B	0	reject switch or recompute the full prefix under B
base -> aLoRA A at position 2	2	fork clean base prefix [0,2), compute invocation and suffix under A
aLoRA A -> base after position 2	2	return to a saved clean fork or recompute the tainted suffix
aLoRA A -> aLoRA B, same boundary	2	share base prefix only; fork distinct adapter suffixes
same aLoRA artifact, different boundary	1 in the exact fixture	activation schedule changes the execution digest
LM-head-only adapter	4 K/V rows	reuse K/V but recompute adapter-specific prompt logits
same semantic adapter relocated to another slot	4	reuse; physical slot identity is not semantic K/V identity

Ordinary LoRA’s conservative full-prefix recompute is deliberate. A more specialized engine could prove a longer layer- or row-specific compatible prefix, but then it must version the architecture graph, preserve the needed intermediate state, and still satisfy M5’s all-layer publication contract. Most engines gain little by turning this edge case into a second cache ABI.

0x08K Design Alternatives

Design	Safety proof	Reuse	Cost / limitation
raw adapter revision in every key	simple and conservative	none across adapter IDs	misses output-only and pre-invocation equivalence
dependency-closed interval digest	architecture graph plus activation schedule	maximal under the proven graph	complex versioning and negative-test surface
aLoRA boundary contract	paper-proven base equality before invocation	clean prefix shared across base and aLoRAs	method-specific training/runtime and tainted suffix handling
full prefix recompute on every switch	no cross-adapter equality needed	none	highest prefill latency and duplicate cache memory
layer-selective recompute	per-layer dependency and intermediate-state proof	potentially more	usually requires hidden-state materialization and a new publication ABI

The recommended default is conservative ordinary-LoRA identity plus the explicit aLoRA exception. General dependency-closed reuse should be added only when the runtime can validate the model graph and test every supported target pattern.

0x08L Adapter/KV Failure Lab

Failure	Why it looks plausible	Required closure
check only k_proj/v_proj targets	Q/O/MLP factors are absent from direct projection	compute architecture dependency closure or reject conservatively
reuse base cache for ordinary Q-only LoRA	current layer K/V shapes and bytes match	downstream K/V can differ; recompute or use proven interval contract
require raw aLoRA ID before invocation	adapter is attached to request	normalize proven inactive interval to base execution digest
omit aLoRA ID after invocation	prefix began under base execution	fork at boundary; suffix is adapter-specific
use first invocation occurrence	token sequence contains a valid marker	pinned PEFT uses the last exact occurrence
activate after the invocation tokens	phrase “after invocation” is ambiguous	pinned implementation starts at the first invocation token
invocation absent silently runs product as base	PEFT’s row-level fallback is shape-safe	reject when the product contract requires specialization
inputs_embeds request silently disables aLoRA	PEFT cannot derive token offsets	require explicit verified offsets or reject
mutate one cache from base -> A -> base	tensor allocation remains valid	keep clean fork; never relabel tainted suffix
adapter revision omitted under same alias	name still matches runtime key	content changed; advance execution digest and cache namespace
slot generation added to semantic prefix key	protects pointer lifetime elsewhere	destroys reuse after harmless relocation; keep it in ownership receipt
K/V match treated as cached-logit match	output-only adapter leaves K/V unchanged	recompute hidden/logit artifact under its own compatibility contract
boundary falls inside full block but whole block is reused	most rows are base-equivalent	stop at legal block/tail frontier owned by M5.3
cache candidate validated before authorization	bytes may never be returned	lookup itself can leak cross-tenant prefix existence; authorize first

0x08M Executable Loop 8 Oracle

The local exact oracle proves only the bounded teaching contract:

direct K adaptation changes layer-0 K from [0,1] to [1,3]
Q-only adaptation leaves layer-0 K/V unchanged but changes layer-1
    K from 6 to 7 and V from 4 to 9/2
LM-head-only adaptation changes logits [3,1] -> [6,0] with unchanged K/V
PEFT-style last-occurrence offsets produce masks [F,F,T,T]
    and [F,F,F,T,T,T]
base, ordinary LoRA, aLoRA, output-only, relocation, position,
    representation, layout, and token compatibility cases close exactly
pre-invocation execution normalizes to base; post-invocation does not
semantic execution fields remain separate from slot/block ownership fields
55 one-field malformed fixtures fail closed with 52 distinct error codes

It executes no Transformer, PEFT model, attention kernel, vLLM cache lookup, GPU allocation, paper benchmark, latency test, throughput test, or model-quality evaluation. The softmax witness proves one dependency path; the conservative graph projection is a rewrite contract for unsupported target combinations, not a claim that pinned PEFT or vLLM implements that general key.

0x09 Scheduler, Fairness, Observability, And Recovery

An adapter-aware scheduler does not replace the serving scheduler built in M5.2. It extends one already-legal iteration plan with new facts:

which immutable adapter artifact the request names
whether its exact generation is resident and published
whether the worker supports every target/rank/layout
how many adapter, basis, temporary, and K/V pages the request reserves
which phase-specific heterogeneous kernel can execute its rows
how much cold-load, rank-padding, run-fragmentation, and collective work it adds

That distinction is the owner contract for this loop.

M5.2 still owns	M6.5 Loop 9 adds
request state and legal row intervals	immutable adapter execution identity
token, sequence, and future-state budgets	adapter/basis residency and published generation
TTFT/TBT clocks and SLO feasibility	cold-load time and adapter-specific readiness
generic continuous batching and chunked prefill	adapter run, rank, target, and basis affinity
client/job/group fairness definitions	projections by adapter class and load eligibility
preemption, resume, and completed-service publication	adapter load rollback and request-generation revalidation

Loop 9 therefore does not claim a new continuous-batching scheduler or a new fairness theorem. It asks a narrower systems question:

Once M5 has declared which work is legal and urgent, how should adapter residency, heterogeneous execution cost, fairness projections, telemetry, and load failure change the decision without weakening M5’s contracts?

0x09A State Added By An Adapter Request

For request $r$ on worker $w$, freeze an adapter execution record:

$$\mathcal{A}_{r,w} = (d_a,g_a,\mathbf{r}_a,\mathcal{T}_a,b_a,s_{a,w},g_{a,w}^{pub},\Gamma_w)$$

where:

Symbol	Meaning
$d_a$	immutable artifact/content digest
$g_a$	requested semantic adapter generation
$\mathbf{r}_a$	module-specific logical-rank vector
$\mathcal{T}_a$	target-module and factor-layout manifest
$b_a$	optional shared-basis/cluster identity from Loop 6
$s_{a,w}$	worker residency state: cold, loading, resident, retiring, failed
$g_{a,w}^{pub}$	atomically published worker generation
$\Gamma_w$	worker kernel/TP/layout capability contract
$g_{catalog}^{pub}$	adapter generation the catalog currently publishes for the request’s alias
$g_r^{lease}$	the $g_{catalog}^{pub}$ that $r$ bound at admission and keeps for its lifetime (0x07H); a later hotswap does not change it

This record is not the whole request. M5 still owns prompt length, decode frontier, K/V state, deadline clocks, cancellation state, and the candidate iteration budget. Adapter state is an additional execution prerequisite.

The scheduler must also retain separate clocks:

request arrival clock          never reset by adapter loading or retry
adapter-load attempt clock     new attempt ID for every reservation/copy
generation-publication clock   advances only after validated copy completion
iteration selection clock      planned work, not completed service
iteration result clock         authoritative completed-service publication

Resetting request age after a cold load makes loading delay disappear from TTFT and can manufacture apparent fairness. Reusing a failed load attempt ID can make stale copy events appear to complete the current generation.

0x09B Five Predicates, Not One Score

A single weighted score hides incompatible logical states. A large affinity bonus can accidentally make an unsupported or stale request look selectable. Use five predicates instead.

1. Adapter feasibility

$$\begin{aligned} Feasible_A(r,w) ={}& [d_{base}(r)=d_{base}(w)] \\ &\land [\mathcal{T}_a\subseteq\Gamma_w.targets] \\ &\land [\mathbf{r}_a\preceq\Gamma_w.rank\_limits] \\ &\land [layout_a\in\Gamma_w.layouts] \\ &\land [g_a=g_r^{lease}] \\ &\land JointReservePossible(r,w) \end{aligned}$$

This predicate consumes the exact artifact checks from Loop 2, joint adapter/K/V reservation from Loop 5, TP placement from Loop 6, and K/V compatibility from Loop 8. The generation test compares $g_a$ with the request’s lease, not with the catalog’s current generation: after a hotswap (0x07F) an in-flight request still holds the old generation, and 0x07H requires it to finish on that generation. A request admitted after the hotswap leases the new $g_{catalog}^{pub}$, so asking for a superseded generation still fails.

2. Adapter readiness

$$Ready_A(r,w,t) = Feasible_A(r,w) \land \big([s_{a,w}=resident]\lor[s_{a,w}=retiring\land Leased_w(r)]\big) \land CopyEventComplete(a,w) \land [g_{a,w}^{pub}=g_a]$$

A cold request can be feasible without being ready. A resident slot with the wrong generation is neither ready nor a cache hit.

$Leased_w(r)$ is true when $r$ already holds a refcounted lease (0x07H) on the slot that serves $d_a$ on worker $w$; $s_{a,w}$, the copy event, and $g_{a,w}^{pub}$ describe that slot. A retiring slot accepts no new binds, but the requests that leased it before a hotswap stay ready on it. After the publication in 0x07F, request-old keeps running on retiring slot 7 with its generation-8 lease while request-new binds generation 9 on slot 8, and the refcount and last-use event keep slot 7 from being reclaimed under it.

3. Execution eligibility

$$Eligible(r,w,t) = Eligible_{M5}(r,w,t) \land Ready_A(r,w,t)$$

Only this intersection may enter an adapter kernel launch. A cold request may instead be load-eligible, which authorizes a prefetch transaction but not a model row.

4. Preference

Preference orders already-legal candidates. It may improve affinity, fairness, or expected cost. It cannot repair a failed predicate.

5. Selection and completion

$$Selected_t \subseteq Eligible_t$$ $$Completed_t = \{r\in Selected_t: Result_t(r)=committed\}$$

Selected is a plan. Completed is a publication event. Adapter prefetch, failed copies, cancelled rows, speculative plans, and recomputation do not become completed tenant service merely because they consumed GPU time.

The distinction is easy to test in one snapshot:

Request	Feasible?	Ready?	Reason
r_hot, resident A generation 7	yes	yes	exact generation and supported rank/targets
r_cold, cold B generation 3	yes	no	load may be scheduled; model row may not
r_peer, resident C generation 5	yes	yes	shares basis 0 with A but retains distinct artifact
r_stale, requests A generation 6	no	no	published generation is 7; holds no generation-6 lease
r_rank, rank 32 on max-rank-16 worker	no	no	backend ABI unsupported

The Loop 9 fixture selects only from r_hot and r_peer; it cannot promote r_cold, r_stale, or r_rank by adding affinity weight.

0x09C The Adapter Cost Is A Vector

The original score sketch collapsed unlike units. Replace it with a cost vector that can be measured and constrained independently:

$$\mathbf{C}_A(r,w,P) = \left( B_{load}, B_{basis}, \Delta N_{runs}, R_{true}, R_{pad}, P_{adapter}, P_{KV}, B_{collective} \right)$$

For an iteration plan $P$:

$$R_{true}(P) = \sum_{i\in rows(P)} r_{a(i),m(i)}$$ $$R_{pad}(P) = \sum_{i\in rows(P)} r^{physical}_{slot(i),m(i)}$$ $$\rho_{rank}(P) = \frac{R_{pad}(P)-R_{true}(P)}{R_{true}(P)}$$

R_true predicts useful low-rank row work. R_pad exposes a fixed-slot backend’s padding overhead. Delta N_runs predicts segmented launch and metadata fragmentation. Load bytes, joint pages, and collective bytes are not interchangeable with either rank ledger.

A conservative local ordering can therefore be lexicographic:

$$K(r,w,t) = \left( U_{M5}(r,t), \frac{W_{tenant(r)}(t)}{\omega_{tenant(r)}}, -Age(r,t), \mathbf{C}_A(r,w,P), t_{arrival}(r), id(r) \right)$$

Lower keys run first. The ordering means:

1. preserve M5’s urgency/deadline class;
2. prefer the protected tenant with less completed weighted service;
3. age requests so affinity cannot defer one forever;
4. minimize adapter load, basis load, run, padding, page, and collective cost;
5. break ties deterministically.

This is a Transformer System teaching policy, not Punica, S-LoRA, VTC, or vLLM. In particular, no VTC bound transfers after adding readiness, loading, prefetch, or this completion-reconciled ordering.

0x09D PetS: Two-Stage Coordinated Batching

PetS predates modern autoregressive multi-LoRA engines, but it exposes the first important scheduling decomposition: shared operators and task-specific operators need different grouping decisions.

For macro-batch $i$, PetS models latency as:

$$T_i = \alpha[N_i,L_i] + \sum_j \beta[pt_{ij},n_{ij},l_{ij}]$$

alpha models the shared operator over total batch size $N_i$ and maximum length $L_i$. beta models PET type $pt_{ij}$, same-task mini-batch size $n_{ij}$, and length $l_{ij}$.

Why the Bellman recurrence appears

Within one task, sort requests by length. If the last mini-batch contains requests $j\ldots i$, every earlier request $1\ldots j-1$ must already be optimally partitioned; otherwise replacing that prefix with a cheaper partition would improve the whole solution. This optimal-substructure argument gives:

$$D^{PET}[i] = \min_{0<j\le i} \left( D^{PET}[j-1] + \beta[pt_i,i-j+1,l_i] \right)$$

After producing same-task mini-batches, sort those mini-batches by maximum length and repeat the argument for shared operators:

$$D^{shared}[i] = \min_{0<j\le i} \left( D^{shared}[j-1] + \alpha\left[ \sum_{k=j}^{i}n_k, L_i \right] \right)$$

The exact Loop 9 toy table gives:

task A lengths [2,4,8]
    optimal PET groups = [a0,a1] + [a2]
    cost = 7 + 9 = 16

task B lengths [3,3]
    optimal PET group = [b0,b1]
    cost = 6

sorted mini-batches = [taskB.mb0(max=3), taskA.mb0(max=4), taskA.mb1(max=8)]
optimal shared groups = [taskB.mb0,taskA.mb0] + [taskA.mb1]
cost = 10 + 9 = 19

PetS then orders PET operators with an operational-intensity proxy and assigns different intensities across streams to reduce resource conflict. But its latency model treats PET operators sequentially while its stream schedule can overlap them; the paper explicitly leaves a more accurate model and online QoS scheduling as future work. PetS therefore motivates hierarchical grouping. It does not prove a decoder K/V scheduler, LoRA cold-load policy, or tenant fairness contract.

0x09E Punica: Consolidation, Async Loading, And Migration

Punica uses a different owner object: a global scheduler places requests among GPU working sets while each GPU invokes the shared base path and heterogeneous LoRA rows.

For a new request, the paper’s candidate workers satisfy:

$$\mathcal{W}_{feasible}(r) = \left\{ w: |WS_w|<B^{max}_w \land KV^{free}_w\ge KV^{need}_r \right\}$$

It then chooses the feasible worker with the largest working set:

$$w^* = \arg\max_{w\in\mathcal{W}_{feasible}(r)} \left(|WS_w|,UUID_w\right)$$

The UUID is a deterministic tie-break in the paper. This consolidation policy keeps busy GPUs busy, lets lightly loaded GPUs drain, and leaves idle GPUs releasable.

The exact toy placement is:

Worker	Working set	Free K/V pages	Feasible for 8 pages?
gpu-0	8	100	yes
gpu-1	12	5	no
gpu-2	8	20	yes

gpu-0 and gpu-2 tie at working-set size 8, so the fixture’s UUID ordering selects gpu-2. The result reconstructs the source rule; it does not establish that consolidation is optimal for another topology, K/V allocator, or SLO mix.

Punica also separates placement from readiness:

adapter absent
    -> issue asynchronous host-to-device copy
    -> continue running the existing working set
    -> join the request only after adapter load completion

Under K/V pressure, the paper migrates the newest request to preserve FCFS, cancels its source execution, discards the newly generated token, and recomputes the prompt plus prior output on the destination rather than copying K/V. That is a paper-specific recovery choice. M5.2 and M5.5 still own whether recompute, preserve, transfer, or reject is legal in another runtime.

0x09F S-LoRA: Clustering, Prefetch, And Early Abort

S-LoRA adopts iteration-level scheduling, keeps the large adapter population in host memory, and loads only adapters needed by the active batch. Three policy choices matter here.

Adapter clustering

Let $A(P)$ be the number of distinct adapters in plan $P$. Prioritizing same-adapter requests can reduce:

$$A(P),\qquad \Delta N_{runs}(P),\qquad B_{load}(P)$$

but a hot adapter can then dominate the running batch. S-LoRA explicitly frames clustering as a throughput/latency/fairness tradeoff and studies cluster size in an ablation. “Fewer adapters per batch” is therefore an optimization objective, not a fairness rule.

Queue-derived prefetch

While batch $P_t$ runs, S-LoRA predicts adapter demand for $P_{t+1}$ from the current waiting queue and uses available GPU memory to overlap transfer with compute:

$$Prefetch_t \subseteq DistinctAdapters(Waiting_t)$$

subject to available memory. The exact selection and benefit depend on the queue and memory policy. The Loop 9 fixture uses an explicitly labelled teaching FCFS first-fit rule:

waiting adapters = [B, B, C, D]
pages            = {B:3, C:4, D:2}
available        = 6
teaching choice  = [B,D] using 5 pages

This reproduces neither S-LoRA’s scheduler code nor a canonical paper algorithm. It merely makes the queue-to-prefetch state transition executable.

Early abort under overload

S-LoRA rejects a request whose estimated prefill can no longer meet its maximum latency:

$$t_{now}-t_{arrival,r}+\widehat{T}_{prefill} > T^{max}_r$$

Its appendix estimates a serviceable count $l$. When the moving arrival estimate $R_1$ exceeds the moving admission estimate $R_2$, it selects the newest serviceable requests; otherwise it selects the earliest. In the exact fixture:

t_now=10, estimated prefill=2, l=2, R1=5, R2=2
q0 arrival=0 -> infeasible
q1 arrival=4 -> feasible
q2 arrival=6 -> feasible
q3 arrival=8 -> feasible
overload choice -> [q3,q2]

Newest-first under overload maximizes the paper’s defined reward under stated assumptions. It can sacrifice old requests. Calling it “fair” would change the protected identity and objective without proof.

0x09G A Local Adapter-Aware Selection Trace

The local policy composes the closed contracts in this order:

def plan_adapter_extension(m5_candidates, worker, completed_service, now):
    classified = []
    for req in m5_candidates:
        adapter = req.adapter_lease  # bound at admission (0x07H), not re-resolved
        feasible = adapter_feasible(req, adapter, worker)  # Loops 2, 5, 6, 8
        ready = feasible and adapter_ready(req, adapter, worker)  # Ready_A: resident, or retiring under this lease
        classified.append((req, adapter, feasible, ready))

    # Cold work may request a load transaction. It is not a model row yet.
    load_candidates = [x for x in classified if x.feasible and not x.ready]
    prefetch = choose_by_m5_urgency_then_age(load_candidates)
    if prefetch is not None:
        begin_loop5_reservation_copy_publish(prefetch)

    executable = [x for x in classified if x.ready]
    executable.sort(key=lambda x: (
        m5_urgency_class(x.req),
        completed_service[x.req.tenant_id] / tenant_weight(x.req.tenant_id),
        -request_age(x.req, now),
        adapter_cost_vector(x, worker),
        x.req.arrival_tick,
        x.req.request_id,
    ))

    extension = pack_without_breaking_m5_budgets(executable)
    return publish_generation_stamped_plan(extension, prefetch)

The first line of defense is still m5_candidates. Adapter logic cannot make an M5-illegal token row legal. The second is the immutable generation snapshot. The third is readiness. Only then does the policy compare completed service, age, and affinity.

The static fixture has two ready requests:

r_hot  tenant completed service = 3, resident adapter A
r_peer tenant completed service = 1, resident adapter C

They share the same M5 urgency class. With a one-sequence budget, the local key chooses r_peer. The cold request is separately eligible for a load decision; the stale-generation and unsupported-rank requests fail closed.

0x09H Completed Service And Adapter-Specific Fairness Projections

Consume M5.2’s completed-service definition. For tenant $u$:

$$W_u[t_1,t_2] = w_p N^{completed\ input}_u + w_q N^{completed\ output}_u$$

Loop 9 may project the same completion events by adapter class $c$:

$$W^{adapter}_c[t_1,t_2] = \sum_{r:\,class(a_r)=c} W_r[t_1,t_2]$$

The class may be a bounded operational grouping such as:

resident vs cold-loaded
rank bucket
phase
target signature
shared-basis cluster

But this projection is not tenant fairness. One adapter may serve many tenants; one tenant may use many adapters. Publishing equal service per adapter can still starve a tenant, and equal tenant service can still leave one cold adapter unloaded.

The readiness pipeline needs two liveness predicates.

Adapter-load starvation:

$$LoadStarved(a,[t_1,t_2]) = LoadEligibleThroughout(a,[t_1,t_2]) \land PublishedLoads_a[t_1,t_2]=0$$

Tenant-service starvation, inherited from M5.2:

$$ServiceStarved(u,[t_1,t_2]) = ExecutionEligibleThroughout(u,[t_1,t_2]) \land CompletedEvents_u[t_1,t_2]=0$$

A cold adapter that never becomes ready is a load-starvation case, not yet an execution-eligible tenant-service starvation case. Keeping those predicates separate reveals whether the failure belongs to prefetch/residency or to batch selection after readiness.

0x09I Six-Tick Starvation Witness

The exact trace contains one resident adapter A, one cold adapter B, one cold request c0, and a new A request at every tick.

Tick	Resident-greedy	Deficit + age + affinity
0	run h0	run h0
1	run h1	run h1
2	run h2	prefetch B; run h2
3	run h3	publish B; run c0
4	run h4	run h3
5	run h5	run h4

Resident greed completes six hot-service units and never publishes B. The exact fixture records LoadStarved(B)=true. It does not claim ServiceStarved(cold tenant)=true, because B never became execution-ready.

The local policy triggers prefetch at age 2, publishes B at tick 3, and then selects the cold tenant because its completed service is 0 versus the hot tenant’s 3. The final ledger is:

resident-greedy:
    completed service = {hot:6}
    load-starved adapters = [B]

deficit + age + affinity:
    completed service = {hot:5, cold:1}
    load-starved adapters = []
    service-starved tenants = []

This proves progress only for one six-tick synthetic trace. It is not a universal wait bound, a VTC theorem, an SLO proof, or a production comparison.

0x09J Prefetch And Admission Compete For Real Pages

Prefetch is speculative memory work. Admission is a commitment to a request. They must not draw independently from the same free-page counter.

For worker $w$:

$$P^{free}_w = P^{total}_w -P^{base}_w -P^{active\ KV}_w -P^{resident\ adapter}_w -P^{temp}_w -P^{reserved}_w$$

A prefetch candidate is safe only if its provisional reservation preserves the already-admitted K/V growth horizon:

$$P^{prefetch}_a \le P^{free}_w -P^{committed\ KV\ growth}_w -P^{safety}_w$$

The load state machine is:

load-eligible request
  -> provisional joint-page reservation
  -> asynchronous copy under attempt_id
  -> digest/layout validation
  -> copy-complete event
  -> atomic pointer + generation publication
  -> request may become execution-ready

Eviction, cancellation, timeout, or prediction changes before publication must release provisional pages. A prefetched adapter that is never used records wasted load bytes and occupancy time, not completed request service.

0x09K Observability Begins At Publication Events

A metric is meaningful only after naming the state transition that publishes it. Loop 9 uses this event spine:

Event	What becomes authoritative	Example metrics
queue_enqueued	original request arrival and identity	queue age, cold-wait histogram
prefetch_reserved	provisional pages owned by one attempt	reserved pages, prediction count
adapter_load_started	transfer attempt began	in-flight copies, requested bytes
adapter_load_completed	copy finished, not yet necessarily published	H2D latency/bytes
adapter_generation_published	validated addresses and generation visible	residency state, cache hit eligibility
iteration_selected	immutable plan chosen	planned rows/runs/rank work; zero service debit
iteration_completed	execution result known	success/failure/cancelled rows
completed_service_debited	M5 service counters advance	tenant and adapter-class completed service
prefetch_released	provisional ownership closed	wasted-prefetch pages/bytes/time

For event $e$, a counter increment is valid only when the event is the declared publication point:

$$Counter_k(t^+) = Counter_k(t^-) + \mathbf{1}[PublishEvent(e)=k]$$

This prevents three common errors:

1. count a planned row as completed service;
2. count copy completion as resident before generation publication;
3. count reserved pages as both free and resident.

Metrics also need denominator and identity contracts:

$$AdapterCacheHitRate = \frac{ResidentGenerationHits} {ResidentGenerationHits+ColdOrStaleMisses}$$ $$PrefetchPrecision = \frac{PrefetchedGenerationsUsedBeforeEviction} {PublishedPrefetchedGenerations}$$ $$LoadAmplification = \frac{AdapterBytesTransferred} {AdapterBytesFirstUsedByCompletedRequests}$$

The denominator must include failed, cancelled, or wasted attempts where the metric’s meaning requires them. Reporting only successful load latency hides failure tail behavior.

Cardinality is an operational contract

Do not attach raw request, tenant, or adapter IDs to default time-series labels. A large adapter catalog can turn one useful metric into millions of series. Keep default labels bounded:

phase
rank_bucket
residency_state
result
stage

Use sampled traces, logs, exemplars, or a bounded top-k/debug endpoint for raw identity. Per-tenant fairness accounting may remain in a state store without becoming an unbounded metrics label.

The exact event fixture publishes two queue arrivals, one adapter load, two selected iterations, two completions, and two service debits. It derives:

load-B-1 latency = 1 tick
completed queue wait = {h0:0, c0:3}
tenant-class completed service = {resident:1, cold:1}
adapter-class completed service = {resident:1, cold-loaded:1}

0x09L Recovery Is A Transaction, Not A Retry Label

Consider a cold adapter D with eight free pages and a three-page load.

tick 0  queue d0; reserve 3 pages for load-D-1      free=5
tick 1  load-D-1 fails; publish zero service
tick 1  rollback load-D-1 reservation               free=8
tick 3  reserve 3 pages for load-D-2                free=5
tick 3  validate and publish generation 2
tick 4  select and complete request; debit service=1
tick 5  release request reservation                 free=8

The original queue arrival remains tick 0. The second attempt gets a new ID and new generation. Failure closes ownership before retry.

def recover_adapter_load(req, attempt, error):
    mark_attempt_failed(attempt, error)       # no completed-service debit
    unpublish_if_never_committed(attempt)
    rollback_joint_reservation(attempt)
    release_copy_events_and_provisional_pins(attempt)

    req.load_attempt += 1
    req.queue_arrival_tick = req.original_arrival_tick  # never reset
    req.required_generation = catalog.current_generation(req.adapter_id)
    return retry_with_backoff_or_reject(req)

Recovery must also handle:

Failure	Required closure
stale generation discovered after planning	invalidate plan; release provisional pins; rebuild metadata
copy fails before publication	rollback reservation; old generation remains authoritative
failure after generation publication	retire through a successor generation; never mutate published addresses in place
request cancels during load	release request pin; keep or cancel shared prefetch by policy; preserve accounting
prefetch prediction becomes obsolete	cancel before publication or publish as ordinary cache state; record wasted prefetch
selected iteration fails	publish zero completed service; preserve M5 semantic frontiers
K/V grows into prefetched pages	forbid independent allocation; joint reservation/conservation must fail closed

0x09M Policy Alternatives And Failure Boundaries

There is no single adapter scheduler independent of product objective.

Policy	Optimizes	Main risk	Required protection
FIFO after readiness	arrival order	cold load blocks may dominate TTFT	async load and capacity admission
resident greedy	zero/low load cost	adapter-load starvation	age or load-progress quota
adapter clustering	fewer active adapters/runs	other adapters wait	bounded cluster window or age override
basis-cluster affinity	shared-basis reuse	cluster dominance	tenant and raw-adapter projections
rank/target bucketing	efficient kernels	long-tail shape classes wait	bucket aging and fallback backend
Punica-style consolidation	high working-set density	hotspot/SLO imbalance	K/V feasibility and scale-out policy
S-LoRA-style early abort	source reward under overload	sacrifices old requests	explicit offered/rejected outcome ledger
completed-service deficit + affinity	tenant service before local efficiency	no inherited theorem after composition	measure gaps/starvation; claim only observed policy

Failure symptoms identify which predicate was collapsed:

Symptom	Likely mistake
stale adapter pointer executes	readiness used adapter alias, not published generation
hot resident adapter monopolizes every tick	affinity had no load-progress/age protection
“fair” adapter distribution but one tenant receives nothing	adapter class was mislabeled tenant identity
load latency absent from TTFT	queue clock reset at load start or completion
success rate looks high during overload	rejected/offered denominator omitted
resident bytes exceed joint pool	prefetch and K/V reservations used independent free counters
fairness counters increase on failed kernels	planned work was debited as completed service
load failure leaks pages	attempt did not close reservation/pin/copy ownership
monitoring backend overloads	raw adapter/tenant/request IDs became metric labels

0x09N Executable Loop 9 Oracle

The local artifact closes a bounded, exact teaching contract:

PetS dynamic programs recover intra-task cost 16 / 6 and macro cost 19
Punica placement filters K/V-infeasible gpu-1 and selects gpu-2 by tie-break
S-LoRA teaching prefetch chooses B,D in 5 of 6 pages
S-LoRA overload example chooses newest feasible q3,q2 under source reward
local snapshot separates feasible [r_cold,r_hot,r_peer]
    from ready [r_hot,r_peer] and selected [r_peer]
resident greed leaves adapter B load-starved across six ticks
age-triggered prefetch publishes B and completes cold request c0 at tick 3
event-derived counters reconcile load latency, queue wait, and completed service
failed load-D-1 restores all pages; load-D-2 completes without resetting age
64 one-field malformed fixtures fail closed with 54 distinct error codes

Run it with:

python3 docs/numinens-evidence/m6-5-multi-adapter-serving-runtime/adapter_scheduler_oracle.py \
  docs/numinens-evidence/m6-5-multi-adapter-serving-runtime/adapter_scheduler_fixture.json \
  --write-report docs/numinens-evidence/m6-5-multi-adapter-serving-runtime/loop9_report.json

The oracle executes no PetS, Punica, S-LoRA, vLLM, GPU allocator, adapter kernel, production scheduler, or paper benchmark. It proves no latency, throughput, SLO-attainment, universal bounded-wait, or fairness theorem. Its exact evidence is limited to the fixture’s Bellman tables, placement snapshot, prefetch/abort examples, six-tick trace, event ledger, and recovery transaction.

0x0A Punica, S-LoRA, PEFT, And vLLM Source-To-Runtime Oracle

The previous sections derived the states a serving runtime needs. We can now ask a harder question:

When Punica, S-LoRA, PEFT, and vLLM all say that they execute LoRA adapters, what is actually the same, what is represented differently, and where does each pinned source stop providing evidence?

The answer cannot be a feature table alone. A library can expose a method named load_adapter, a server can expose a route named load_lora_adapter, and a kernel wrapper can accept several weight pointers, while the three operations still have different identity, residency, activation, publication, and retirement semantics.

Loop 10 therefore uses four evidence layers:

Layer	Question	Evidence type
algebra	does a row receive the intended low-rank delta?	exact rational-arithmetic oracle
representation	how are row identity, rank, scale, and weight location encoded?	revision-pinned source trace
lifecycle	when is an artifact registered, resident, selectable, and retired?	source-observed state transitions plus explicit unknowns
performance	how fast is the path on a concrete system?	not measured by this loop

The pinned revisions are Punica v1.1.0, S-LoRA’s archived final commit, PEFT v0.19.1, and vLLM v0.12.0. The source observations below are not promises about later releases.

Transformer System reconstruction. The four lanes share one normalized mathematical transaction. They do not share an artifact format, metadata ABI, lifecycle API, or demonstrated safety protocol.

Source: Punica, Figure 2, reproduced from the official open-access paper. Each runner keeps a shared base model and K/V cache while the scheduler manages request and adapter placement. This is Punica’s paper architecture, not a generic PEFT, S-LoRA, or vLLM memory map.

0x0AA One Transaction, Four Non-Interchangeable Contracts

Consider one linear target module. The base path has five rows:

$$X\in\mathbb{Q}^{5\times3}, \qquad Y^{base}\in\mathbb{Q}^{5\times4}.$$

Rows use the adapter map:

row                  0    1    2       3       4
logical adapter      A    A    B    __base__    B

Both adapters have rank two, but they have different scales:

$$s_A=\frac{1}{2}, \qquad s_B=\frac{3}{2}.$$

For row $i$, define $a(i)$ as its adapter identity. The common semantic contract is:

$$Y_i = Y_i^{base} + \begin{cases} s_{a(i)}\left(X_iA_{a(i)}^\top\right)B_{a(i)}^\top, & a(i)\in\{A,B\},\\ 0, & a(i)=\texttt{\_\_base\_\_}. \end{cases}$$

This formula does not specify a runtime representation. The same mapping is encoded four ways in the fixture:

Backend surface	Identity seen by execution	Fixture representation
Punica	adapter pointer position plus contiguous segment	permutation [0,1,2,4,3], segments [0,2,4]
S-LoRA	compact adapter index plus indexed weight locations	slots [0,0,1,-1,1], rank/scale/start/length/location tables
PEFT	adapter name for each item on the first input axis	['A','A','B','__base__','B']
vLLM	positive request ID translated to active GPU slot	IDs [11,11,22,-1,22], slots [1,1,0,-1,0]

The owner contract is deliberately narrow. Loop 10 proves that these four declared representations reconstruct the same five outputs after explicit normalization. It does not run any framework or prove that their APIs, schedulers, kernels, retirement paths, or performance are interchangeable.

0x0AB Punica: Pointer Vectors, Contiguous Segments, And A Fixed Rank

The pinned Punica source makes its kernel contract concrete.

Weight storage and pointer indirection

LoraWeight allocates:

$$W_A^{store}\in\mathbb{R}^{L\times r\times d_{in}}, \qquad W_B^{store}\in\mathbb{R}^{L\times r\times d_{out}}.$$

PEFT-style $B\in\mathbb{R}^{L\times d_{out}\times r}$ is transposed during copy. BatchedLoraWeight then stores device vectors of integer addresses:

$$p_A[j]=address(W_{A,j}^{store}), \qquad p_B[j]=address(W_{B,j}^{store}).$$

The kernel therefore receives no dense tensor with an adapter axis. It receives pointer vectors and a segment boundary vector.

Why the segment vector appears

Suppose requests are stably permuted so every row for adapter $j$ is contiguous. If adapter $j$ owns $n_j$ rows, define:

$$S_0=0, \qquad S_{j+1}=S_j+n_j.$$

Then segment $j$ is the half-open interval $[S_j,S_{j+1})$. In the exact fixture:

permuted original rows = [0,1 | 2,4 | 3]
adapter segments       = [ A  |  B  | base]
kernel segment vector  = [0,2,4]

The base-only row is outside the two adapter segments. After the segmented kernel, output rows are scattered back through the inverse permutation.

BatchedLlamaLoraWeight constructs the cumulative segment vector, but it also sets:

self.rank = weights[0].q.lora_rank

The q/k/v calls pass that one scalar rank with every pointer vector. That is strong source evidence for a fixed-rank invocation in this pinned wrapper. It is not evidence that the Punica idea, paper, or all later implementations forbid variable-rank adapters.

The scale boundary is external

The pinned PEFT converter extracts and stacks only A/B tensors, enforces one rank, and does not visibly read adapter_config.json or carry (lora_alpha/r). The kernel wrapper likewise has no per-adapter scale argument on this path. Therefore the oracle does not pretend raw PEFT artifacts can be compared directly. It declares:

$$\widetilde B_a=s_aB_a$$

before constructing Punica’s stored B payload. This is a Transformer System normalization contract, not behavior attributed to the converter.

# Semantic reconstruction, not an import of Punica.
for j, adapter_id in enumerate(adapter_order):
    for packed_row in range(segment[j], segment[j + 1]):
        original_row = permutation[packed_row]
        Y[original_row] += X[original_row] @ A[adapter_id].T @ B_scaled[adapter_id].T

The source shows the low-rank invocation. Punica’s paper owns cluster scheduling, asynchronous loading, and migration claims discussed in 0x09. The pinned model/kernel tree does not, by itself, prove a production registration, lease, generation-publication, or retirement protocol.

0x0AC S-LoRA: Variable-Rank Indexed State And A Publication Gap

S-LoRA moves the adapter identity out of contiguous pointer segments and into indexed resident state. The archived InferAdapter stores:

adapter_dirs   compact index -> artifact directory
idx_map        artifact directory -> compact index
a_start[j]     start in the adapter-location vector
a_len[j]       number of occupied cells
a_loc[...]     physical allocator locations
a_scaling[j]   per-adapter alpha/r scale
prefetch_tag   protected membership for the current prefetch generation

For the attention-only archived path, one adapter consumes (4r) allocator cells: q, k, v, and o low-rank pieces. Thus:

$$a\_len_j=4r_j, \qquad a\_start_j=\sum_{u<j}a\_len_u.$$

This is why variable rank is a length-table problem rather than a single scalar rank argument. The exact fixture keeps both ranks equal to two so all four backends can share one numeric transaction, but S-LoRA’s representation still carries a rank and scale per adapter.

Loading mutates metadata and issues copies

The pinned load_adapters does the following:

scan missing adapters
-> first-fit against available allocator cells for prefetch
-> allocate all selected cells
-> append dirs/start/length/location/scale metadata
-> issue A/B copies, optionally on prefetch_stream

The ordering matters. A directory may already exist in idx_map while the prefetch stream is still performing its copies. The class exposes no explicit adapter-local completion event or READY(generation) bit. The router awaits load RPCs and uses global CUDA synchronization on selected non-prefetch batch transitions, but this is not the same as the event-scoped publication protocol derived in 0x07.

The correct reading is therefore:

$$Indexed(a)=1 \not\Rightarrow CopyComplete(a)=1 \not\Rightarrow SafeForAnyInFlightGeneration(a)=1.$$

That is not a claim that the archived system necessarily races in its intended execution path. It is an evidence ceiling: the inspected class does not prove a general adapter-local publication theorem.

Retirement compacts every dependent index

offload_adapters frees removed locations, compacts every surviving metadata tensor, and rebuilds idx_map. If a consumer retained an old compact index across this operation, it could name a different adapter after compaction. A legal runtime must either quiesce those consumers, version the mapping, or lease the old generation. The pinned source span does not prove which generalized protocol should be used.

Finally, the archived LoraTpPartAdapter constructor asserts world_size == 1. S-LoRA’s paper describes tensor-parallel designs, but this pinned constructor cannot be cited as executable TP evidence. Paper contract and archived-code contract remain separate.

0x0AD PEFT: Named Model Slots And First-Axis Mixed Batches

PEFT owns a different layer of the stack. Its primary object is an adapted model with named adapter modules, not a K/V-aware multi-tenant server.

Loaded is not active

In PEFT v0.19.1, load_adapter(name) adds a named adapter to the model, while set_adapter(name) changes the active adapter. The two calls encode two states:

$$Loaded(name) \not\Rightarrow Active(name).$$

This distinction is useful, but it is process-local model state. It does not provide admission, GPU-slot capacity, request generation, K/V compatibility, or in-flight retirement semantics.

Mixed batches group the first input axis

The pinned LoRA layer accepts adapter_names and requires:

$$|adapter\_names|=|X|_{axis\ 0}.$$

It rejects this route when adapters are merged and rejects mixed adapter names when DoRA is enabled. __base__ identifies rows that receive no adapter delta.

The implementation computes the base layer once, groups row indices by name, and applies one low-rank branch per named sub-batch:

result = base_layer(x)
for adapter_name in unique(adapter_names):
    if adapter_name == "__base__":
        continue
    rows = [i for i, name in enumerate(adapter_names) if name == adapter_name]
    result[rows] += B[adapter_name](A[adapter_name](x[rows])) * scaling[adapter_name]

Let $G_a$ gather rows named $a$, and let $G_a^\top$ scatter them back. The grouped update is:

$$Y = Y^{base} + \sum_{a\in\mathcal A} G_a^\top \left[ s_a(G_aX)A_a^\top B_a^\top \right].$$

Because every row belongs to exactly one named group or __base__, gathering and scattering only reorders independent row computations. This proves the same row-wise algebra. It does not prove that Python sub-batching is a high-throughput token scheduler.

Hotswap has a model-compatibility contract, not a serving lease contract

PEFT’s compiled hotswap helper can pad ranks and tensorize scaling before compilation. Its state-dict path dry-runs key compatibility, zeroes missing old keys, updates scaling, and copies or pads new tensors. The pinned source still states important limits: target-module sets must remain compatible, the rough config check is LoRA-specific, and restoring the original model may require a reload.

None of these operations supplies the generation and lease protocol from 0x07. Mutating a model slot in place while requests use it is a separate serving problem.

0x0AE vLLM: Request Identity, Registered Cache, Active Slots, And Token Metadata

vLLM v0.12.0 connects a request-facing identity to bounded resident slots and kernel metadata. That path has several distinct state transitions.

Request identity is composite

The pinned LoRARequest carries:

lora_name
lora_int_id > 0
lora_path
optional base model and long-context metadata

The class explicitly notes that global integer-ID uniqueness is required but not enforced there. Equality and hashing use the name. Therefore a robust catalog must validate name, integer ID, path/content revision, base revision, and generation before a request reaches a worker.

Registered cache and active GPU slots are different

LoRAModelManager maintains:

_registered_adapters   CPU-side adapter objects, bounded by max_cpu_loras
_active_adapters       GPU-active identities, bounded by max_loras
lora_index_to_id       active GPU slot -> positive LoRA integer ID

The configuration enforces:

$$max\_cpu\_loras\ge max\_loras\ge1.$$

Activation finds a free slot, records the adapter ID in lora_index_to_id, resets each layer’s slot, and copies the adapter’s true rank extent. Registration success alone does not establish an active slot, and an active slot alone does not establish which token rows use it.

Scaling is absorbed before execution

The pinned LoRALayerWeights.optimize() path performs:

$$B_a\leftarrow s_aB_a, \qquad s_a\leftarrow1.$$

The layer then calls the indexed shrink/expand wrapper with scale one. This is mathematically equal because scalar multiplication commutes with the final linear expansion:

$$s_a(XA_a^\top)B_a^\top = (XA_a^\top)(s_aB_a)^\top.$$

It is not artifact-byte parity with PEFT or S-LoRA; it is an execution-time normal form.

Max-rank slots preserve algebra through zero tails

For true rank $r_a$ and configured physical rank $R\ge r_a$, vLLM allocates fixed slot extents. Define:

$$A_a^{pad} = \begin{bmatrix} A_a\\ 0_{(R-r_a)\times d_{in}} \end{bmatrix}, \qquad \widetilde B_a^{pad} = \begin{bmatrix} s_aB_a & 0_{d_{out}\times(R-r_a)} \end{bmatrix}.$$

Then:

$$(XA_a^{pad\top})\widetilde B_a^{pad\top} = (XA_a^\top)(s_aB_a)^\top.$$

The zero tails preserve the intended output. They do not prove that padding is free in memory traffic, workspace, or kernel work. Loop 4 owns the true-rank versus padded-rank cost ledger.

Request IDs become stable kernel groups

The exact fixture intentionally places B in slot 0 and A in slot 1:

lora_index_to_id       = [22,11]
token request IDs      = [11,11,22,-1,22]
token slot mapping     = [ 1, 1, 0,-1, 0]
stable sorted row IDs  = [ 3, 2, 4, 0, 1]
active slot IDs        = [-1,0,1]
counts                 = [ 1,2,2]
starts                 = [ 0,1,3,5]

-1 is the no-adapter group. The stable ordering preserves original row order inside each slot group. These metadata are related to Punica’s segments, but they are not the same ABI: vLLM first resolves request identity to an active slot, then derives indexed kernel groups.

Dynamic API removal is not enough evidence for worker retirement

The inspected load endpoint locks by adapter name, allocates an integer ID, asks the engine to add/preload the adapter, and only then publishes the API name mapping. The inspected unload endpoint removes that name from the API registry under its lock. That source span does not itself call a worker-cache retirement path or establish that old in-flight requests hold leases.

Therefore the strongest justified statement is:

$$RemovedFromAPINameMap(a) \not\Rightarrow WorkerPayloadRetired(a) \not\Rightarrow NoInFlightUser(a).$$

This is an evidence boundary, not a claim that every complete vLLM execution leaks adapters or is unsafe. The exact engine/control-plane path must be audited before asserting retirement semantics.

0x0AF Exact Algebraic Normalization

The oracle uses integer matrices and rational scales, so no floating-point tolerance can hide a metadata error. For each adapter:

$$Z_{i,a}=X_iA_a^\top\in\mathbb{Q}^{1\times2},$$ $$\Delta Y_{i,a}=s_aZ_{i,a}B_a^\top\in\mathbb{Q}^{1\times4}.$$

The derived delta table is:

Row	Adapter	$\Delta Y_i$
0	A	[3, 3/2, 0, -3/2]
1	A	[-1/2, -5/2, 3, 1]
2	B	[9, 9/2, 0, -9/2]
3	base	[0, 0, 0, 0]
4	B	[6, 3/2, 3, -3/2]

Adding the fixture’s nonzero base output gives:

row 0  [13,   3/2, 1,  -5/2]
row 1  [-1/2, 5/2, 1,   2  ]
row 2  [12,  15/2, 3,  -3/2]
row 3  [-1,  -1,  -1,  -1  ]
row 4  [8,   11/2, 9,  13/2]

The four reconstruction proofs are now explicit:

1. Punica: permute rows into contiguous adapter segments, evaluate each segment with externally pre-scaled B, and inverse-permute outputs.
2. S-LoRA: use each row’s compact adapter index to recover its rank, scale, and indexed locations, then evaluate the true-rank factors.
3. PEFT: gather rows by adapter name, evaluate one named sub-batch at a time, and scatter into the base result.
4. vLLM: map request IDs to active slots, evaluate zero-padded A and scale-absorbed B, and restore rows through stable metadata.

Each produces the exact table above. Changing one segment, one scale, one slot, one rank tail, or one base-row sentinel changes the output or fails validation.

0x0AG Lifecycle State Is Not A Shared API

Place the systems on common coordinates without pretending their concrete states are identical:

Coordinate	Punica v1.1.0 source	S-LoRA archived source	PEFT v0.19.1	vLLM v0.12.0
validate artifact	converter checks tensor names/shapes and one rank	router/config records rank and artifact directory	config/state loading into named adapter	PEFT helper validates supported features, rank, bias, targets
register	construct LlamaLoraWeight and batched pointer vectors	directory enters router catalog/compact map	load_adapter(name) adds module state	adapter object enters registered CPU cache
make resident	caller supplies device-resident tensors	allocator pages and A/B copies	parameters reside in the model process	activation copies into bounded GPU slot
publish mapping	caller creates contiguous segments	router batch transition; no adapter-local event in inspected class	set_adapter or per-item adapter_names	request ID -> slot -> per-token kernel metadata
execute	segmented shrink + expand	indexed variable-rank prefill/decode path	base once plus named sub-batch deltas	indexed shrink/expand over active slots
retire	not proved in pinned model/kernel tree	free pages, compact arrays, rebuild idx_map	model API, not a serving lease protocol	API name removal does not itself show worker retirement

The lifecycle comparison exposes two general rules.

First, identity changes coordinate. A name, path, compact index, pointer-list position, positive request ID, active slot, and token slot are not synonyms. Every transition needs a checked map.

Second, publication changes authority. Allocation, metadata insertion, copy submission, copy completion, active-slot installation, row mapping, and request admission are separate events. Collapsing them into loaded=True recreates the races derived in 0x05 and 0x07.

0x0AH What Parity Does And Does Not Mean

Parity layer	Loop 10 result	Why
normalized LoRA output	exact	rational oracle returns the same five rows
logical row-to-adapter assignment	exact	each representation reconstructs A,A,B,base,B
raw artifact bytes	not equal	layout, transpose, padding, and scale placement differ
metadata ABI	not equal	segments, compact page tables, names, and slot metadata differ
API surface	not equal	model library, archived server, kernel wrapper, and serving engine own different calls
lifecycle semantics	not equal	registration, activation, publication, and retirement differ
in-flight hotswap safety	not proved	no common lease/generation theorem is present in all pinned paths
latency or throughput	not measured	no framework, GPU, or benchmark is executed

This distinction prevents two opposite mistakes. We should not dismiss the systems as incomparable: they do share a low-rank row algebra that can be tested exactly. But we should not turn equal algebra into operational parity: serving correctness depends on all the surrounding state.

Compress then Serve remains a fifth design family rather than a fifth backend in this oracle. It changes the adapter collection representation through shared bases and adapter-specific cores. Loop 6 already derives that algebra and placement. Mixing it into the four-way runtime fixture would test a different question: approximation/reconstruction error as well as dispatch.

0x0AI Source-To-Runtime Failure Boundaries

Failure	Locally observable symptom	Why shape checks may still pass	Fail-closed action
Punica mixed ranks passed under first-adapter rank	wrong pointer stride or ignored true rank	every pointer still has an address	split by rank or use a verified variable-rank ABI
Punica PEFT scale omitted	numerically wrong adapter strength	A/B shapes and rank are valid	carry config-derived scale or pre-scale under a recorded transform
Punica rows not contiguous by adapter	segment applies wrong weights	total row count and segment endpoint match	validate every packed row against segment identity
S-LoRA idx_map treated as copy completion	row can observe incomplete payload	compact index exists	require an authoritative completion/publication event
S-LoRA compact index retained across offload	row names a moved adapter	index remains in integer range	version or rebuild all consumers after compaction
archived S-LoRA code cited as TP execution	unsupported world size enters constructor	paper describes TP	keep paper and code evidence separate
PEFT newly loaded slot assumed active	request receives old/default adapter	named tensors are loaded	activate explicitly or provide per-item names
PEFT mixed names used on merged/DoRA layer	unsupported semantics	input and output shapes match	reject before forward as pinned source does
PEFT hotswap used as serving publication	in-flight rows can cross generations	target tensor shape can be padded	drain, lease, or generation-publish at runtime level
vLLM integer ID collision	two artifacts can alias a runtime identity	both IDs are positive	enforce catalog uniqueness with artifact generation
vLLM request ID mapped to stale slot table	valid slot selects wrong adapter	slot is in bounds	publish mapping and active-slot generation atomically
nonzero max-rank tail	padded coordinates add an unintended delta	physical shape is correct	zero/reset slot before bounded copy
API unload treated as worker retirement	stale residency or in-flight user remains	name disappears from list endpoint	audit engine retirement and lease completion separately

The exact negative corpus mutates these and related boundaries one field at a time. It includes identity, rank, scale, segment, page, name, slot, padding, metadata, lifecycle, parity, and claim-ceiling failures.

0x0AJ Executable Loop 10 Oracle

The evidence bundle contains:

backend_runtime_fixture.json
    one 5x3 input, one 5x4 base result, two rank-2 adapters,
    four representations, lifecycle matrix, parity matrix, claim ceiling

backend_runtime_oracle.py
    exact Fraction-based LoRA evaluation
    Punica segment reconstruction
    S-LoRA compact-index reconstruction
    PEFT named gather/scatter reconstruction
    vLLM ID/slot/padded-rank reconstruction

invalid_backend_runtime_fixtures.json
    95 one-field malformed fixtures
    95 distinct fail-closed error codes

Run it with:

python3 docs/numinens-evidence/m6-5-multi-adapter-serving-runtime/backend_runtime_oracle.py \
  docs/numinens-evidence/m6-5-multi-adapter-serving-runtime/backend_runtime_fixture.json \
  --write-report docs/numinens-evidence/m6-5-multi-adapter-serving-runtime/loop10_report.json

The oracle imports no framework and executes no PyTorch, CUDA, GPU allocator, kernel, server, scheduler, dynamic API, or benchmark. It proves no latency, throughput, memory-footprint, distributed-execution, API-compatibility, production-lifecycle, or in-flight hotswap-safety claim. Its exact evidence is the normalized five-row algebra, declared representation transforms, pinned source observations, lifecycle non-equivalence, and fail-closed fixture set.

0x0B Operational Metric Registry

Section 0x09K defines when state becomes authoritative. This section lists the metric surface. Every implementation must attach each metric to one of those publication events, document its denominator, and keep default labels bounded.

Scheduler and liveness metrics:

adapter_requests_offered_total{phase,rank_bucket}
adapter_requests_admitted_total{phase,rank_bucket}
adapter_requests_rejected_total{reason,phase}
adapter_requests_unfinished{phase,residency_state}
adapter_load_eligible_wait_seconds{rank_bucket}
adapter_execution_ready_wait_seconds{phase,rank_bucket}
adapter_load_starvation_intervals_total{rank_bucket}
adapter_service_starvation_intervals_total{phase,rank_bucket}
adapter_prefetch_requests_total{result}
adapter_prefetch_precision_ratio
adapter_prefetch_wasted_bytes_total{reason}
adapter_completed_service_units{phase,rank_bucket}
adapter_completed_service_gap{phase,rank_bucket}
adapter_affinity_overrides_total{reason}
adapter_plan_invalidations_total{reason}
adapter_generation_replans_total{stage}

These counters do not establish fairness by themselves. A starvation metric needs a protected identity, eligibility predicate, interval, and completion event. adapter_load_starvation protects load-eligible adapter state; adapter_service_starvation protects execution-eligible tenant or client state. Keep them separate.

Required adapter lifecycle and kernel metrics:

adapter_load_requests_total
adapter_load_failures_total
adapter_load_latency_seconds
adapter_h2d_bytes_total
adapter_gpu_resident_bytes
adapter_host_resident_bytes
adapter_cache_hits_total
adapter_cache_misses_total
adapter_evictions_total
adapter_pinned_count
adapter_distinct_per_iteration
adapter_rows_per_iteration
adapter_prefill_rows_per_iteration
adapter_decode_rows_per_iteration
adapter_prefill_requests_per_iteration
adapter_decode_requests_per_iteration
adapter_rank_histogram
adapter_rank_histogram_by_phase
adapter_true_rank_row_values
adapter_padded_rank_row_values
adapter_rank_padding_ratio
adapter_contiguous_runs_before_reorder
adapter_contiguous_runs_after_reorder
adapter_kernel_seconds
adapter_kernel_seconds_by_phase
adapter_metadata_build_seconds
adapter_gather_scatter_seconds
adapter_batch_fragmentation_ratio
adapter_tp_collective_bytes_total
adapter_tp_collective_seconds
adapter_hotswap_seconds
adapter_slot_generation_mismatch_total
adapter_basis_gpu_resident_bytes
adapter_basis_cache_hits_total
adapter_basis_cache_misses_total
adapter_basis_load_latency_seconds
adapter_basis_version_mismatch_total
adapter_cluster_rows_per_iteration
adapter_joint_core_kernel_seconds
adapter_residency_state_count{state}
adapter_page_reservations
adapter_pages_reclaimable
adapter_retirement_wait_seconds
adapter_publication_generation
adapter_publication_failures_total{stage}
adapter_copy_events_inflight
adapter_compensating_rollbacks_total{result}
adapter_victim_restore_bytes_total

Joint adapter/KV metrics:

kv_bytes_by_adapter
joint_memory_reservation_failures_total
adapter_load_blocked_by_kv_total
kv_admission_blocked_by_adapter_total
prefix_cache_hits_by_adapter
prefix_cache_adapter_mismatch_total
joint_pool_pages{class,state}
joint_pool_conservation_failures_total
joint_admission_shortfall_pages
joint_growth_reserve_pages

Without joint metrics, adapter cache pressure and KV pressure can blame each other while the operator sees only generic OOMs.

Raw adapter, request, or tenant identifiers are not default Prometheus-style labels. Preserve them in sampled traces, structured logs, exemplars, or bounded top-k debugging views. The fairness/accounting state store may retain exact identity without exporting one time series per catalog entry.

0x0C Failure Injection

The cases below are the human-readable attack surface. They are intentionally more concrete than a release checklist: each line names one event that an operator, scheduler, loader, or kernel test can inject. The executable evidence bundle contains 553 one-field malformed fixtures across nine mechanism families. Section 0x0D does not replace either corpus. It gives those detailed cases a typed recovery index so a failed detector has one consequence, one repair owner, and one release gate.

Test cases:

load adapter with wrong base revision
load rank above kernel maximum
submit mixed ranks to a fixed-rank segmented kernel
leave nonzero values in a reused slot's padded A tail
leave nonzero values in a reused slot's padded B tail
count one adapter row per prefill request instead of per token
set doff before the final prefill row
reuse adapter segments as attention request boundaries
select the first rather than last prefill logit row
reorder prefills without rebuilding the scheduler-output inverse map
load adapter with missing target module
cancel request during adapter transfer
evict adapter while queued request holds reservation
evict adapter whose generation is still named by a live launch snapshot
publish adapter page table before the copy-complete event
publish adapter with one factor page or target-module entry missing
reuse retired victim pages, then fail copy before publication
rollback overwritten victim without a verified host source
release reservation bitmap but leave an adapter or KV owner attached
hotswap slot while kernel is in flight
mix two adapter revisions under one cache key
switch K/V-changing adapter mid-request
reuse base cache for a Q-only adapter without downstream dependency closure
reuse aLoRA suffix cache after switching back to base
select the first rather than last aLoRA invocation occurrence
omit invocation tokens and silently serve base behavior
reuse K/V-compatible LM-head cache as if prompt logits also matched
place physical slot generation in the semantic prefix key
force host-to-device transfer slowdown
fill GPU with adapters, then grow decode KV
fill GPU with KV, then request a cold adapter
inject malformed scaling metadata
load a compressed core with the wrong basis version
evict a shared basis while requests in its cluster are in flight
add an adapter to a frozen cluster without rebuilding decomposition metadata
let resident affinity override a failed M5 legality predicate
schedule a feasible cold request before its copy event and generation publish
keep selecting a hot resident adapter while never issuing a load for an aged cold adapter
call load-eligible cold wait execution-eligible tenant starvation
call adapter-class progress per-tenant fairness
debit completed service when an iteration is selected rather than committed
debit completed service for prefetch, recomputation, cancellation, or failed execution
reset request arrival time after adapter load or retry
reuse one load attempt ID across rollback and retry
reserve prefetch pages independently of admitted K/V growth pages
publish copy completion as residency before pointer-table generation publication
count only successful loads in the load-latency denominator
attach raw adapter, request, and tenant IDs to every time-series metric
apply S-LoRA newest-first overload abort and call it fair aging
transfer VTC's service bound to the local readiness/affinity composition

Expected behavior should be explicit:

reject
queue
recompute
preempt
evict
fallback
or fail closed

Silent use of a base model or a different adapter is not a safe fallback unless the product contract explicitly permits it.

0x0D Failure Lab

Failure	Missing systems concept	Symptom
one kernel launch per adapter per layer	heterogeneous batching absent	decode overhead rises with adapter cardinality
adapter alias, path, or integer ID treated as immutable content	artifact digest absent	same identifier resolves to different factor bytes or target map
one scalar rank stored for a rank-heterogeneous artifact	module manifest collapsed	shape-correct loading fails or silently pads/truncates a module
logical factor orientation used as physical kernel layout	conversion ABI absent	valid tensor dimensions feed the wrong kernel axes
physical slot generation inserted into semantic prefix key	ownership and compatibility conflated	safe cache reuse is destroyed after harmless adapter relocation
stale slot or pointer generation accepted because adapter name matches	launch snapshot absent	old metadata dereferences republished factor storage
one global generation counter compared across state owners	generation namespaces collapsed	unrelated updates appear ordered while stale owner state survives
prefill adapter ID repeated once per request	request identity not expanded to token rows	tail prompt rows execute the wrong adapter or no adapter
adapter run reused as attention boundary	projection and sequence metadata conflated	cross-request attention or invalid K/V writes
doff disagrees with final prefill pointer	phase manifest stale	rows enter the wrong attention/KV path
prefill request reorder omits inverse sample map	packed and scheduler coordinates conflated	correct logits are returned to the wrong request
runtime chooses first prefill row rather than last	sampling-row map absent	next token ignores most of the prompt
mixed rank sent to pinned fixed-rank Punica wrapper	backend rank ABI ignored	out-of-bounds factor reads or numerically wrong deltas
padded A or B tail retains prior slot bytes	slot reset incomplete	lower-rank adapter inherits another adapter’s update
configured maximum rank treated as logical rank	semantic and physical rank conflated	cost, residency, and artifact validation are wrong
adapter cache fills all free HBM	KV and adapter policies independent	requests fail during KV growth
entire adapter counted as rank pages	one [R,H] tensor confused with all factors/modules/layers	admission under-reserves device slots
registered adapter treated as GPU-resident	host catalog and active slot collapsed	request launches through an absent pointer table
adapter published before copy event and digest validation	publication fence absent	kernel reads partial or stale factor rows
LRU victim selected before ref/pin/event checks	ranking substituted for eligibility	active adapter storage is overwritten
retired victim pages overwritten and bitmap merely released on copy failure	compensating rollback omitted	catalog claims the old adapter is resident over corrupted bytes
adapter and KV pages share allocator and ownership key	physical pool confused with semantic ownership	cross-request KV reuse or adapter pages freed with one request
growth reserve omitted from joint admission	admission horizon incomplete	decode reaches a capacity cliff after successful prefill
pointer generation updated separately from page table	visibility is not atomic	launch combines new identity with old addresses
all ranks padded to maximum	rank heterogeneity ignored	high wasted FLOPs and bandwidth
adapter factors sharded independently of the base projection	tensor-parallel placement contract missing	wrong global update or extra collectives
low-rank all-gather launched separately when fusion was assumed	collective schedule not implemented	small-message latency dominates adapter work
aggressive adapter bucketing	SLO ignored	queueing dominates kernel savings
adapter affinity implemented as one unbounded weighted bonus	feasibility/readiness and preference collapsed	stale, cold, or unsupported request appears selectable
cold request called execution-eligible before generation publication	load eligibility and model-row eligibility collapsed	fairness ledger blames selection for a residency failure
resident-only greed has no age/load-progress override	load liveness absent	cold adapter remains feasible but is never published
adapter-class service reported as tenant fairness	protected identities conflated	one tenant can starve while every adapter bucket progresses
iteration selection debits completed service	plan and result publication collapsed	failed/cancelled rows make fairness counters advance
adapter-load retry resets request arrival	load and request clocks conflated	TTFT and starvation wait are understated
failed load keeps provisional reservation	recovery ownership not closed	free-page ledger leaks and later admission fails
prefetch and K/V each read the same free-page count	no joint reservation serialization	both plans succeed and overcommit one physical pool
raw adapter ID used as default metric label	observability cardinality unbounded	telemetry memory and query cost scale with catalog size
S-LoRA early-abort reward called fairness	objective and protected identity changed	newest-first overload sacrifice is hidden by terminology
VTC theorem attached to local affinity policy	theorem assumptions changed	a synthetic trace is presented as a service bound
hotswap overwrites in-flight slot	lifetime fence absent	nondeterministic mixed-adapter output
adapter revision absent from KV key	cache provenance incomplete	exact-reuse corruption
adapter target names treated as the cache dependency graph	indirect Q/O/MLP effects are omitted	downstream K/V rows are reused under the wrong hidden state
raw aLoRA ID included in every pre-invocation block	inactive execution normalized incorrectly	safe base-prefix sharing is lost
aLoRA ID omitted from post-invocation blocks	one request began with a base prefix	adapter-tainted suffix is reused by base or another adapter
output-head K/V equality treated as prompt-logit equality	cache object types collapsed	base logits are returned under adapter semantics
first invocation occurrence used instead of last	activation implementation not pinned	wrong rows execute adapter factors and receive the wrong cache identity
missing invocation silently accepted as specialized result	framework fallback mistaken for product policy	requested adapter behavior is absent without an admission failure
arithmetic unmerge repeated	finite-precision accumulation ignored	base weight drifts
cold adapter load omitted from TTFT	residency not modeled	tail latency surprises
engine documentation treated as timeless	version contract omitted	unsupported API or kernel path
compressed core loaded with the wrong shared basis	collection artifact provenance omitted	shape-correct but numerically unrelated adapter update
compression ratio treated as throughput gain	three-stage operator and shared rank omitted	lower residency but unchanged or worse TPOT
scheduler groups only by adapter ID	shared-basis affinity omitted	repeated basis loads and fragmented joint kernels
scheduler groups only by basis cluster	queueing/SLO tradeoff omitted	hot clusters starve unrelated requests

0x0DA A Failure Record Needs Five Coordinates

A failure name alone is not an operational contract. stale adapter, for example, could mean stale catalog bytes, a stale physical slot, a stale pointer table, a stale cache key, or an old request lease. Those failures have different detectors and different repair owners. M6.5 therefore indexes every release failure by five semantic coordinates:

1. identity:     which state owner and generation became inconsistent?
2. invariant:    what relation had to remain true?
3. detector:     which authoritative observation proves the violation?
4. consequence: what can become wrong if execution continues?
5. recovery:     who repairs the state, and which test blocks release?

The machine record keeps category, mechanism loop, and test surface as explicit fields around those five coordinates:

$$F_i = (id_i,\ell_i,c_i,I_i,D_i,C_i,O_i,T_i,G_i),$$

where $\ell_i$ is the mechanism loop, $c_i$ the failure category, $I_i$ the invariant, $D_i$ the detector, $C_i$ the consequence, $O_i$ the recovery owner, $T_i$ the executable test surface, and $G_i$ the release gate. A record is not closed merely because its detector fired. It is closed only when the named owner restores the invariant and the associated test returns to green.

This distinction prevents two unsafe shortcuts:

• a metric is not automatically a detector unless it is bound to an authoritative publication or completion event;
• a fallback is not automatically recovery unless it restores the declared semantic state or the product contract explicitly permits degraded service.

0x0DB Failure Families And Alarm Ownership

The release index contains 36 typed failures: four for every mechanism loop from Loop 2 through Loop 10. The table is deliberately compact; Sections 0x01-0x0A and the executable fixtures retain the tensor values, state-machine steps, source spans, and one-field mutations.

Loop	Typed failure	Required invariant	Detector	Consequence	Recovery owner
2	identity_alias_collision	alias resolves to one immutable artifact digest, base revision, and target manifest	catalog/manifest reconciliation	request executes different factors under the same name	identity registry
2	module_rank_manifest_collapse	every target module preserves its own rank, axes, scale, dtype, and shard	module-shape ledger	valid-looking tensor is interpreted under the wrong ABI	artifact validator
2	generation_namespace_collapse	catalog, slot, pointer, batch, cache, and RNG generations remain namespaced	owner-generation snapshot	unrelated updates hide stale owner state	launch validator
2	slot_semantic_identity_conflation	physical slot identity stays outside semantic adapter and prefix identity	relocation-invariance check	harmless movement invalidates reuse or stale slot aliases semantics	cache identity owner
3	row_permutation_not_bijective	gather and scatter maps are inverse over every packed row	permutation oracle	outputs return to the wrong request row	batch packer
3	segment_identity_drift	each contiguous segment contains only rows for its declared adapter	row-to-segment reconciliation	one adapter’s factors update another adapter’s rows	segmented-kernel planner
3	scale_orientation_drift	factor orientation and scale encoding reconstruct the declared LoRA delta	rational tensor oracle	shape-correct branch computes another update	adapter operator owner
3	combine_metadata_loss	origin row, local row, adapter, slot, and scale metadata survive execution	scatter metadata audit	base and adapter outputs combine at wrong coordinates	batch packer
4	phase_row_count_drift	prefill contributes prompt-token rows and ordinary decode one row per request	mixed-phase row ledger	metadata lengths disagree with execution rows	iteration planner
4	rank_tail_nonzero	coordinates beyond true rank are zero in every padded slot	slot-tail scan	a low-rank adapter inherits prior slot bytes	slot loader
4	sampling_inverse_missing	packed sample rows map bijectively back to scheduler request order	output inverse-map oracle	correct logits are returned to the wrong request	scheduler/output bridge
4	attention_boundary_alias	attention request boundaries remain independent of adapter runs and phase split	indptr/segment comparison	cross-request attention or invalid K/V writes	attention batch owner
5	registered_called_resident	catalog registration, page reservation, copy, validation, publication, and activity remain distinct	residency state-machine check	launch dereferences absent or partial factors	residency manager
5	publication_before_copy_validation	copy completion, digest/layout validation, and pointer publication are ordered	event-and-generation receipt	kernel reads partial or stale payload	publication owner
5	eviction_eligibility_bypassed	victim has zero refs, pins, reservations, and incomplete use events before reuse	eligibility predicate audit	live storage is overwritten	eviction manager
5	rollback_owner_leak	failed admission restores victim bytes and releases every provisional adapter/K/V owner	page conservation oracle	allocator leaks or catalog names corrupted data	transaction coordinator
6	tp_factor_placement_drift	adapter factors follow the base projection’s declared tensor-parallel ownership	shard reconstruction oracle	global update is numerically wrong	TP adapter operator
6	collective_schedule_omitted	every required low-rank collective has an owner, byte count, order, and completion event	collective trace	hidden communication or stale partial result	distributed runtime
6	compressed_basis_version_mismatch	adapter core and shared basis carry one compatible immutable version	collection-manifest check	shape-correct reconstruction denotes another adapter	collection loader
6	compression_ratio_called_speedup	storage, transfer, reconstruction, collective, and kernel costs remain separate ledgers	operator/cost reconciliation	lower bytes are advertised as unmeasured throughput	release evaluator
7	arithmetic_unmerge_called_rollback	exact rollback reloads immutable base plus tested adapter lineage	digest comparison after unmerge	finite-precision base drift accumulates	lifecycle manager
7	hotswap_without_generation_lease	each in-flight request retains one immutable adapter generation	request/slot lease audit	one sequence mixes old and new factors	request lifecycle owner
7	retiring_slot_reclaimed_early	refcount is zero and last-use event is complete before slot reuse	retirement fence	new payload overwrites in-flight storage	residency manager
7	postpublication_recovery_called_rollback	pre-publication abort and post-publication replacement use different generation protocols	transaction-state audit	consumers retain receipts for state silently rewritten in place	publication owner
8	kv_dependency_closure_omitted	cache compatibility includes every direct and downstream path that can change stored K/V	architecture dependency closure	prefix cache reuses state from another model function	M5.3/M6.5 cache owner
8	alora_interval_boundary_drift	last invocation occurrence, first active token, and post-activation taint are pinned	interval mask oracle	factors execute on wrong rows or cache identity changes too early/late	aLoRA integration owner
8	prefix_identity_incomplete	semantic adapter digest and activation interval enter exact prefix identity	prefix-key audit	cross-adapter exact-reuse corruption	M5.3/M6.5 cache owner
8	prompt_logit_equivalence_overclaimed	K/V equality, hidden-state equality, prompt-logit equality, and future sampling stay distinct	output-object compatibility matrix	base logits are returned under adapted semantics	API/cache owner
9	feasibility_readiness_collapsed	legal, resident, copy-complete, generation-published, and preferred are separate predicates	scheduler predicate trace	cold or unsupported work appears executable	adapter scheduler
9	load_starvation_hidden	load-eligible age and execution-eligible service use separate clocks and protected identities	starvation interval oracle	a cold adapter can wait forever while hot work progresses	liveness controller
9	service_debited_before_commit	fairness service advances only after successful result publication	completion-ledger audit	cancelled or failed work receives phantom service	fairness accountant
9	recovery_observability_incomplete	attempt IDs, preserved request age, page restoration, and zero failed-service debit are observable	recovery trace	retries hide latency and leak ownership	recovery coordinator
10	backend_representation_normalized_away	Punica, S-LoRA, PEFT, and vLLM metadata/lifecycle contracts remain non-interchangeable	four-backend source oracle	one backend ABI is implemented as if universal	backend integration owner
10	source_claim_ceiling_exceeded	source observation, paper claim, local exact evidence, and unexecuted claim remain distinct	source-span/claim audit	static reading is presented as runtime proof	evidence owner
10	lifecycle_parity_overclaimed	equal normalized output does not imply equal publication, retirement, or in-flight safety	lifecycle matrix	unsafe unload/hotswap behavior inherits false confidence	lifecycle/release gate
10	numeric_parity_called_performance_parity	algebraic equality remains separate from latency, memory, throughput, and API compatibility	benchmark firewall	unmeasured deployment claim enters release notes	release evaluator

The six alarm families are not interchangeable:

Failure category	Primary loops	Alarm surface	Repair boundary
identity and state	2	immutable manifests and generation receipts	registry, manifest, launch snapshot
tensor and kernel	3, 4, 6	exact tensor, permutation, rank-tail, shard, and collective oracles	packer, kernel, TP operator
residency and publication	5, 7	ownership conservation, copy events, leases, and retirement fences	residency/lifecycle transaction
cache and lifecycle	8	dependency closure, activation interval, semantic prefix identity	M5.3 plus M6.5 adapter provenance
scheduler and observability	9	readiness, starvation clocks, completed service, recovery receipts	scheduler and accounting state
evidence and release	10, 11	source ceilings, parity scope, link/figure/visual closure	evidence and release gate

The malformed fixtures remain mechanism-local because they do not share one runtime ABI. Loop 11 reconciles their counts and test surfaces without pretending that a pointer mutation, a scheduler mutation, and a source-ceiling mutation can be executed by one validator:

Mechanism evidence	One-field fixtures	Executable test surface
Loop 2 runtime identity	30	test_runtime_identity_manifest.py
Loop 3 segmented tensor execution	46	test_segmented_tensor_oracle.py
Loop 4 mixed phase and rank	60	test_mixed_phase_rank_oracle.py
Loop 5 residency and admission	59	test_residency_admission_oracle.py
Loop 6 TP and collection compression	58	test_tp_collection_oracle.py
Loop 7 lifecycle and hotswap	86	test_lifecycle_hotswap_oracle.py
Loop 8 adapter/KV/prefix compatibility	55	test_adapter_kv_prefix_oracle.py
Loop 9 scheduler and recovery	64	test_adapter_scheduler_oracle.py
Loop 10 backend runtime contracts	95	test_backend_runtime_oracle.py
Total	553	nine independent mechanism oracles

$$N_{\text{malformed}} =30+46+60+59+58+86+55+64+95 =553.$$

0x0DC Complete Source, Link, Figure, And Visual Surface

Publication closure has a different unit from technical correctness. A correct kernel section can still ship with a moving source alias, a broken local route, an orphaned image, a stale digest, or an unreviewed source figure. Loop 11 therefore performs set equality rather than spot checks:

article external URLs == publication source-registry URLs
article local post routes == existing Markdown owners
article image routes == cumulative visual-provenance routes
retained source figures == article routes with matching asset digests
all 53 A22 figures == explicit defer-to-owner decisions for M6.5

The static inventory after this loop is:

Surface	Closed inventory	What closure proves
source families	21	every local, primary, and official family has a claim ceiling
exact external URLs	45	every published URL has an explicit family, revision policy, evidence file, and claim ceiling
local article routes	8	every M5/M6/A22 cross-reference resolves to source Markdown
cumulative source-trace records	127	every mechanism loop’s source and synthesis evidence remains addressable
A22 figure decisions	53	M6.5 reuses none of the figures owned by M6.3/M6.4/M6.9
M6.5 source-figure decisions	13	every candidate became retain, replace, or defer
retained M6.5 source figures	6	every retained paper figure has a route, asset, and digest
article image routes	29	every route has non-empty alternative text and provenance
responsive native pairs	10	each loop-owned mechanism visual has desktop and mobile geometry

The registry contains 45 exact external URLs. Static SVG validation checks a root viewBox, role="img", title, description, non-empty alternative text, asset existence, and digest equality. It does not prove font loading, MathJax placement, browser layout, theme contrast, or mobile overflow. Those are rendered observations and remain Loop 12.

0x0DD Static Release Candidate And Rollback Gate

Let each static closure bit be true only when its exact registry or oracle returns no mismatch:

$$R_{\mathrm{static}} =F_{\mathrm{failure}} \land S_{\mathrm{source}} \land L_{\mathrm{local\ route}} \land G_{\mathrm{figure}} \land V_{\mathrm{visual}} \land B_{\mathrm{boundary}}.$$

Publication requires two additional evidence classes:

$$R_{\mathrm{publish}} =R_{\mathrm{static}} \land R_{\mathrm{browser}} \land R_{\mathrm{live\ route}}.$$

Loop 11 may set $R_{\mathrm{static}}=1$. It must keep $R_{\mathrm{browser}}=R_{\mathrm{live\ route}}=0$, and therefore $R_{\mathrm{publish}}=0$. The candidate state is awaiting_loop12, not published.

Release coordinate	Loop 11 state	Why
parent source digest	known	the Loop 10 article SHA is recorded
candidate source digest	known	every Loop 11 artifact binds the patched article
failure/source/figure/visual static closure	exact	local validators reconcile the complete declared sets
browser acceptance	not executed	requires normalized desktop/mobile, light/dark rendering
live external-route acceptance	not executed	static URL provenance is not network availability
publication allowed	no	two required acceptance bits are still false
rollback digest	known	the previous source identity is recorded
retrievable rollback artifact	not claimed	a digest is not an artifact locator or executable rollback

This release state is intentionally conservative. It prevents a common audit error in which all files exist becomes the page renders, or a prior SHA becomes rollback is operational without a retrievable immutable bundle.

0x0DE Executable Loop 11 Oracle

The static closure bundle contains:

failure_closure_manifest.json
    36 typed failure records across Loops 2-10

publication_source_registry.json
    one record for every exact external URL published by the article

publication_surface_fixture.json
    expected source, route, figure, visual, fixture, and boundary sets

publication_surface_audit.py
    fail-closed set and digest reconciliation

invalid_publication_surface_fixtures.json
    one-field mutations of the publication contract itself

release_state_fixture.json
    static candidate state and explicit Loop 12 blockers

Run the bounded validator with:

python3 docs/numinens-evidence/m6-5-multi-adapter-serving-runtime/validate_loop11.py
python3 -m unittest \
  docs/numinens-evidence/m6-5-multi-adapter-serving-runtime/test_loop11_publication_surface.py

The oracle reads local Markdown, JSON, Python test surfaces, PNG/SVG assets, and source digests. It opens no browser, performs no live HTTP request, launches no framework or GPU kernel, and reproduces no paper benchmark. Loop 12 owns formula rendering, responsive geometry, theme evidence, console cleanliness, and live route acceptance.

0x0DF Final Formula, Rendering, Accessibility, Boundary, And Rollback Closure

The final audit keeps five publication surfaces separate because none can stand in for another:

Surface	Exact Loop 12 closure	What remains outside the claim
source	228 source-ordered display formulas, the complete fenced-code and table surface, 29 image routes, 45 exact external URLs, and eight local article routes retain their Loop 11 identities	formal proof of every derivation, future availability of third-party hosts, or reproduction of a paper experiment
generated route	every source math node is paired with one completed MathJax rendering and no MathJax error under each normalized profile	compatibility with every browser, font override, assistive technology, or zoom policy
responsive images	ten picture elements select their mobile source at the mobile viewport and their desktop fallback at the desktop viewport; all remaining figures keep their declared single source	perceptual equivalence under every display, color-vision condition, or user stylesheet
browser and local routes	desktop/mobile and light/dark profiles load every article image and local owner route, preserve heading and anchor identity, contain article-owned overflow, and emit no article-owned browser error	live external URL health, production adapter execution, GPU kernels, memory, latency, throughput, model quality, or tenant isolation
boundaries and release	concrete M5.1—M5.3, M6.3, M6.4, M6.7, M6.9, and M8 owners remain distinct; a deterministic rollback recipe reconstructs the Loop 11 source digest before the candidate is promoted	completion of the still-dynamic M6.9 owner or a production deployment rollback

The Loop 11 state remains an immutable historical record: awaiting_loop12, browser acceptance false, and publication false. Loop 12 does not rewrite that earlier gate. It adds a final release record whose browser and local-route checks are true and whose verdict is publishable_with_declared_scope. Live external availability remains false because the browser audit does not turn third-party network health into a publication invariant.

Rollback is executable at the article-source boundary rather than inferred from a remembered digest. The release bundle names the start and end markers of this Loop 12-only span, removes that span from a temporary copy, and requires the reconstructed bytes to hash exactly to the Loop 11 candidate. It never mutates the working article during validation. This proves one narrow rollback: final M6.5 article source to the immediately preceding static candidate. It does not prove an application, model, adapter, cache, or infrastructure rollback.

The final closure therefore changes publication evidence, not the technical mechanism. It adds no formula, benchmark, source-reported result, serving-policy recommendation, or framework/GPU claim. The formula registry, source registry, mechanism oracles, typed failure records, and figure dispositions retain their earlier claim ceilings.

0x0E Boundary To M8

M6.5 defines the runtime state of live adapters. M8 decides whether a deployment should instead:

merge one adapter
quantize a merged artifact
keep a quantized base plus live adapters
distill adapter behavior into another model
compress a large adapter collection
choose a different serving engine or hardware target

Routed, shared-factor, post-hoc-composed, and continual adapter families add another layer above this runtime. M6.9 owns their routing granularity, training provenance, factor-sharing geometry, lifecycle order, and task-time growth. Those methods still inherit M6.5’s residency, batching, fencing, and K/V provenance contracts.

0x0F Publish Gate

Check	Status
Single-adapter execution, heterogeneous batching, state lifetimes, residency, and KV prerequisites are explicit	pass
Request and worker adapter state are explicit	pass
Base, artifact, semantic-adapter, local-kernel, request, cache/prefix, and slot identities are separate	pass
Module-specific rank, scale, logical axes, factor digests, and local TP shard are immutable manifest fields	pass
Prefix compatibility excludes request/slot/RNG ownership while inheriting semantic adapter identity through the M5 cache ABI	pass
Catalog, request, slot, pointer-table, batch-plan, cache, and RNG generations remain namespaced	pass
The synthetic manifest and 30 negative fixtures reconcile logical, local, and cluster adapter bytes	pass
Shared base GEMM and heterogeneous adapter branch are separated	pass
Naive grouping and segmented kernels are compared	pass
Scheduler, request, active, segment-local, and output row coordinates are separated	pass
The exact 8-row fixture reconciles gather, shrink, expand, scatter, base-only rows, and additive output	pass
Contiguous Punica intervals and indexed vLLM metadata are not conflated	pass
All 46 malformed Loop 3 tensor, pointer, scale, and generation fixtures fail closed	pass
Prefill and decode adapter shapes are distinguished	pass
Mixed-iteration row count is derived from prefill token counts plus one row per ordinary decode request	pass
Prefill indptr, phase doff, adapter row mapping, and scheduler-output inverse mapping are distinct	pass
The exact five-request fixture permits one adapter run to cross doff without granting cross-request attention	pass
Last-prefill-row and one-token-decode sampling coordinates are derived and restored to scheduler request order	pass
Exact-rank, phase-split, rank-bucketed, and zero-padded paths produce the same rational adapter delta	pass
Logical rank, configured/physical rank, and active intermediate width are separate fields	pass
Padded A and B tails are proven zero and the true-versus-padded rank ledger is explicit	pass
Pinned Punica fixed-rank, S-LoRA per-adapter-rank, and vLLM max-rank-slot contracts are not conflated	pass
All malformed Loop 4 phase, row, rank, padding, and sampling fixtures fail closed	pass
Punica SGMV shrink/expand semantics and segment metadata are explicit	pass
Loop, Gather-BMM, and segmented-kernel alternatives are compared	pass
Attention request boundaries and adapter row segments are kept separate	pass
S-LoRA vector-page layout and object lifetime differences are explicit	pass
Adapter residency and cold-load TTFT are quantified	pass
Adapter and KV memory are governed jointly	pass
Primitive vector-page bytes, pinned S-LoRA paired-slot bytes, and complete-adapter page demand are not conflated	pass
Registered, reserved, loading, validated, published, active, idle, and retiring residency states are distinct	pass
Refcount, pin count, queued reservation, in-flight event, and live generation jointly determine eviction eligibility	pass
The exact 16-slot fixture reconciles 1,024 payload bytes and an 8-slot adapter-plus-KV admission	pass
Victim eligibility, victim ranking, and physical page reuse are separate contracts	pass
Copy completion and digest/layout validation happen before one generation-stamped publication point	pass
Pre-publication copy failure performs compensating victim reload rather than only releasing a bitmap	pass
Unified, separate-pool, fixed-slot, host-LRU, load-per-batch, and shadow-copy alternatives are compared	pass
All malformed Loop 5 geometry, ownership, identity, victim, reservation, publication, and rollback fixtures fail closed	pass
Rank heterogeneity is treated as a kernel/scheduler concern	pass
Tensor-parallel factor placement and low-rank communication are derived	pass
Collection-level joint decomposition includes parameter, operator, basis-version, and scheduler consequences	pass
Merge, unmerge, and hotswap have different safety contracts	pass
In-flight slot generation and fencing are explicit	pass
Live and merged LoRA equivalence is derived under explicit orientation, scale, target, arithmetic, and dtype assumptions	pass
Binary16 arithmetic unmerge is proven finite but non-reversible, so immutable base reload owns exact rollback	pass
PEFT layer-local safe merge, whole-model merge/unload, compiled hotswap, and vLLM registration contracts are not conflated	pass
Shadow copy, copy-event completion, digest/layout validation, and atomic catalog publication are ordered explicitly	pass
Old and new requests keep immutable generation leases rather than re-resolving an alias during decode	pass
Retiring slot reclamation requires both refcount zero and last-use event completion, with generation bumps preventing ABA reuse	pass
Pre-publication rollback and post-publication recovery are different generation protocols	pass
All 86 malformed Loop 7 source, tensor, identity, ordering, reclamation, rollback, and claim-ceiling fixtures fail closed	pass
Direct K/V adaptation and indirect Q/O/MLP-to-downstream-K/V dependencies are separated by an architecture closure	pass
The exact Q-only witness leaves layer-0 K/V unchanged while changing layer-1 K/V	pass
K/V compatibility, cached-hidden compatibility, prompt-logit compatibility, and future sampling are distinct	pass
aLoRA’s last-invocation offset, first-token activation, pre-invocation proof, post-invocation taint, training mask, and pinned PEFT limits are explicit	pass
The interval execution digest normalizes proven inactive aLoRA rows to base while binding active rows to adapter content and schedule	pass
Semantic prefix fields exclude request, slot, pointer, block, and sampling ownership fields	pass
Mid-request ordinary LoRA, base/aLoRA, cross-aLoRA, output-only, and relocation cases have different legal closure	pass
Prefix mismatch, pre-publication suffix failure, and post-publication recovery preserve immutable M5 receipts	pass
All 55 malformed Loop 8 source, tensor, activation, dependency, prefix, runtime, and claim-ceiling fixtures fail closed	pass
Generic legal iteration plans, deadline clocks, admission, fairness identities, and preemption remain M5.2 owners	pass
Adapter feasibility, readiness, preference, selection, and completed publication are distinct predicates	pass
Adapter cost remains a vector of load, basis, run, true/padded rank, page, and collective work rather than one dimensionless score	pass
PetS’s two-stage Bellman partition and operator schedule are reconstructed with their sequential-model and QoS limits	pass
Punica’s feasible-worker consolidation, asynchronous adapter load, and recompute migration retain paper-specific scope	pass
S-LoRA’s clustering, queue-derived prefetch, and overload early-abort reward are not mislabeled fairness	pass
Completed tenant service and adapter-class projections remain different protected identities	pass
Adapter-load starvation and execution-eligible tenant-service starvation use different predicates	pass
The exact six-tick trace compares resident greed with age-triggered prefetch and completed-service deficit without claiming a theorem	pass
Prefetch and K/V growth consume one joint reservation/conservation ledger	pass
Metrics bind to authoritative queue, load, generation, result, service, and release events with bounded default labels	pass
Failed load recovery preserves request age, uses a new attempt/generation, debits zero failed service, and restores all pages	pass
All 64 malformed Loop 9 source, policy, paper, liveness, telemetry, recovery, and claim-ceiling fixtures fail closed	pass
One exact five-row transaction is reconstructed through Punica, S-LoRA, PEFT, and vLLM metadata without conflating their lifecycle contracts	pass
All 95 malformed Loop 10 representation, lifecycle, parity, and claim-ceiling fixtures fail closed	pass
Thirty-six typed failures assign every mechanism loop four explicit invariants, detectors, consequences, recovery owners, and test surfaces	pass
All 553 one-field malformed fixtures reconcile across nine independent mechanism families	pass
All exact external URLs, local routes, source spans, A22 dispositions, M6.5 source figures, image routes, and visual digests close statically	pass
The Loop 11 release state remains recorded as awaiting_loop12; static closure does not retroactively claim browser rendering, live external availability, publication, or executable rollback	pass
Loop 12 reconciles formulas, four browser profiles, responsive selection, local routes, article accessibility, concrete owner boundaries, and a source-level rollback into publishable_with_declared_scope	pass

0xFF References

• Transformer Series A22 - LoRA
• M6.3 - LoRA Foundations
• M6.4 - Low-Rank Adaptation Design Space
• M6.7 - Low-Rank Theory, Subspaces, and Dynamics
• M6.9 - Routed, Shared, and Continual Adapters
• LoRA: Low-Rank Adaptation of Large Language Models, v2
• PetS: A Unified Framework for Parameter-Efficient Transformers Serving
• Punica: Multi-Tenant LoRA Serving, v1
• Punica official implementation, v1.1.0
• Punica v1.1.0 BatchLenInfo
• Punica v1.1.0 mixed prefill/decode model path
• Punica v1.1.0 multi-LoRA TUI batch construction
• S-LoRA: Serving Thousands of Concurrent LoRA Adapters, MLSys 2024
• S-LoRA official implementation, archived revision
• S-LoRA archived unified allocator
• S-LoRA archived adapter load/offload path
• S-LoRA archived mixed-rank prefill/decode integration
• S-LoRA archived variable-rank prefill kernels
• S-LoRA archived variable-rank decode kernels
• Compress then Serve, v4
• Compress then Serve, final ICML 2025 / PMLR 267
• Hugging Face PEFT v0.19.1 LoRA reference
• Hugging Face PEFT v0.19.1 hotswap reference
• Hugging Face PEFT v0.19.1 hotswap implementation
• Hugging Face PEFT v0.19.1 LoRA layer merge and unmerge
• Hugging Face PEFT v0.19.1 tuner merge, unload, and wrapper replacement
• Activated LoRA: Fine-tuned LLMs for Intrinsics, v5
• Hugging Face PEFT v0.19.1 activated-LoRA guide and cache-reuse patterns
• Hugging Face PEFT v0.19.1 aLoRA configuration
• Hugging Face PEFT v0.19.1 aLoRA mask, offset, and forward implementation
• Hugging Face PEFT v0.19.1 aLoRA offset and activation tests
• M5.1 - Autoregressive Serving State
• M5.2 - KV Memory Management and Continuous Batching
• M5.3 - KV Layout and Exact Reuse
• vLLM v0.12.0 LoRA reference
• vLLM v0.12.0 OpenAI adapter registration and unload surface
• vLLM v0.12.0 LoRA request identity
• vLLM v0.12.0 LoRA configuration
• vLLM v0.12.0 worker LoRA residency manager
• vLLM v0.12.0 registered and active LoRA managers
• vLLM v0.12.0 max-rank resident linear slots
• vLLM v0.12.0 stable LoRA kernel metadata
• vLLM v0.12.0 indexed LoRA shrink operator
• vLLM v0.12.0 indexed LoRA expand operator
• CUDA C++ Programming Guide 13.0: asynchronous execution and events