M3.11 — Representation Readout and Task-Head Inference

0x00 Contextual Rows Are Not Yet A Task Result

M3.10 closed one autoregressive request: a selected vocabulary token is appended, model state advances, and the loop repeats until a stopping contract closes the output. That is only one way to consume Transformer state.

An encoder-style call can expose all valid contextual rows at once:

input IDs and metadata
    -> embeddings and positional state
        -> full-visibility Transformer blocks
            -> contextual rows H [B,S,D]

This is an explicit input contract. Tokenization is complete, M2.2 has already constructed the initial rows, and M3.1-M3.7 have already contextualized them. M3.11 receives H plus row/task metadata; it does not reopen token-ID lookup, input-table initialization, positional composition, or block internals.

The tensor is useful, but it is not yet any of these objects:

a sequence label
a real-valued prediction
one label per source token
an extractive answer span
a fixed-width search vector
a query-document relevance score

Those outputs require a readout contract. The contract states which rows are valid, which rows are selected or pooled, which learned map consumes them, which score type is produced, and which decision or similarity rule follows. Changing any one of those objects can preserve tensor rank while changing the meaning of the model.

Owner question

How do complete contextual rows become pooled vectors, sequence or token predictions, extractive spans, and retrieval scores without confusing the readout architecture with its training objective or serving system?

This article follows the path:

contextual rows H [B,S,D] + valid-row/task metadata
    -> row selection or pooling
        -> task projection or representation normalization
            -> typed logits, spans, vectors, or pairwise scores
                -> task decision and release artifact

It does not assume that “use the [CLS] token” is a universal algorithm. It does not call a cosine score a probability. It does not treat a dual encoder as a cheaper cross encoder with unchanged semantics. It derives the object that each method changes.

Why the generative endpoint was not enough

M3.8 owns the learned map from hidden width to an ordered vocabulary axis:

$$H_{b,s,:}\in\mathbb{R}^{D} \longrightarrow Z_{b,s,:}\in\mathbb{R}^{V}.$$

That head is tied to token identity. A sequence classifier instead maps one readout vector to a task label axis of width $C$:

$$r_b\in\mathbb{R}^{D_r} \longrightarrow z_b\in\mathbb{R}^{C}.$$

A token classifier preserves the row axis:

$$H\in\mathbb{R}^{B\times S\times D} \longrightarrow Z^{token}\in\mathbb{R}^{B\times S\times C}.$$

An embedding system may produce no class logits at all:

$$H,M \longrightarrow e\in\mathbb{R}^{D_e} \longrightarrow s(e_q,e_d)\in\mathbb{R}.$$

The final scalar is a similarity or relevance score under a declared geometry, not automatically a calibrated probability. These paths share a contextual backbone while defining different output axes, persistent state, training objectives, and reusable runtime artifacts.

Reconstructed owner map. Every branch consumes contextual state, but the branch chooses a different row transformation and output identity. A shared backbone does not make the heads interchangeable.

Historical pressure: representations made task heads small, not trivial

Earlier task systems often trained a task-specific encoder and output module together. The pretraining-and-fine-tuning program changed that allocation: learn a contextual encoder from broad text, then adapt it with a comparatively small output layer for a downstream task. BERT is a primary example. Its paper describes a deep bidirectional encoder representation and reports adaptation to classification and extractive question answering with little task-specific architecture beyond output layers. This is an externally verified statement about that system, not a claim that every task needs only one linear layer. (Devlin et al., 2018)

The change exposed a new interface:

pretrained contextual state
    -> task-specific readout
        -> task-specific loss and decision

The interface later split again. Sentence-BERT uses siamese or triplet structures so sentence representations can be computed separately and compared with cosine similarity. DPR independently encodes queries and passages so document vectors can be precomputed. ColBERT also encodes the two sides independently but retains multiple contextual vectors and delays fine-grained interaction until a lightweight scoring stage. These systems do not differ only in speed; they move the interaction boundary. (Reimers and Gurevych, 2019, Karpukhin et al., 2020, Khattab and Zaharia, 2020)

Reading the inherited architecture figure precisely

Source-local figure from A4. The left branch is the important starting point here: an encoder returns $l$ contextual rows together. The box labeled Output (l tokens) does not specify a pooling rule, task head, score type, or training objective. The decoder labels describe incremental generation and belong to M3.10, not to this article’s readout path.

The missing specification can be written as a typed function:

$$R_\phi: \left( H\in\mathbb{R}^{B\times S\times D}, M\in\{0,1\}^{B\times S}, \mu \right) \mapsto Y,$$

where:

H   contextual rows from a declared layer or layer mixture
M   valid-row mask in the readout's polarity
mu  task metadata: offsets, segments, special rows, label map, prompts
phi learned or configured readout state
Y   a typed result, not an anonymous tensor

The codomain of $R_\phi$ is the method. For example:

Readout	Row transformation	Output object
sequence classification	select or pool valid rows	logits [B,C]
regression	select or pool valid rows	values [B,R]
token classification	preserve aligned valid rows	logits [B,S,C] plus offset map
extractive QA	preserve rows, project two score channels	start/end logits [B,S]
single-vector embedding	pool, optionally project and normalize	vectors [B,D_e]
cross encoder	encode a joint pair, read one score	pair scores [B] or [B,C]
bi-encoder	encode sides independently	reusable vectors plus similarity
late interaction	retain multiple vectors per side	token-level similarity reduction

Architecture, objective, score, and decision are separate contracts

Four layers of terminology are frequently collapsed:

Layer	Question	Example
backbone architecture	Which rows and parameters produce H?	full-visibility encoder
readout architecture	Which rows become the task representation?	masked mean plus projection
training objective	Which target relation shapes the representation?	cross-entropy or contrastive loss
inference decision	How is the score consumed?	argmax, threshold, nearest neighbor

An encoder-only architecture does not imply [CLS] pooling. Mean pooling does not imply contrastive training. L2 normalization does not imply cosine is the only admissible score. A binary sigmoid threshold does not imply that 0.5 is calibrated for the release population.

Claim status

Status	Meaning in this article
Source-local	Preserved from A2/A4/A5 or a neighboring Transformer System owner; provenance is not independent verification.
Externally verified	Supported by a primary paper or pinned official implementation for the scoped claim.
Source-reported	A paper’s benchmark or systems number; not reproduced and not assumed portable.
Rewrite interpretation	A boundary, name, or pedagogical synthesis introduced here.
Synthetic contract test	Checked by a deterministic fixture/oracle; not a benchmark.
Uncertain	Plausible but not sufficiently verified for a settled claim.

Owner boundary

M3.11 consumes:

M2.1: sequence validity, segment, and offset metadata
M2.2: ID-addressed initial state and the lookup/contextual/readout terminology handoff
M3.1-M3.7: contextual row computation
M3.9: supervision/update transaction when training a readout

M3.11 owns:

contextual-output and valid-row readout contract
pooling and row-selection formulas
sequence, token, span, regression, and multilabel heads
representation normalization, contrastive objectives, similarity, and decision typing
cross-, bi-, and late-interaction retrieval readouts
readout runtime, evaluation, migration, release, and rollback

M3.11 defers:

vocabulary projection and token policy                  -> M3.8
general labels/loss/backward/optimizer transaction      -> M3.9
autoregressive feedback, cache, stopping, and streaming -> M3.10
physical operation and traffic accounting              -> M4
multi-request scheduling and service policy             -> M5
fleet evaluation, canarying, production rollback        -> M8.10
modality-specific encoders, fusion, and readout          -> X1
ANN index construction, sharding, and search internals   -> retrieval systems

The forbidden duplication is equally important. This article will not re-derive input lookup or self-attention, repeat the complete training loop, teach sampling, or turn approximate-nearest-neighbor implementation into an M3 subtopic.

Article map

The remaining sections build the contract in dependency order:

immutable contextual/readout manifest
    -> valid rows, offsets, segments, and special-row identity
        -> pooling derivations and backward paths
            -> sequence/token/span task heads
                -> representation geometry and decisions
                    -> cross/bi/late interaction
                        -> concrete representation-family transformations
                            -> training and runtime reconciliation
                                -> evaluation, migration, release, rollback

The durable rule is:

Name the row set, write the readout function, type the output score, and identify which state can be reused before comparing methods or runtimes.

0x01 The Readout Is An Immutable Artifact Contract

A backbone checkpoint is insufficient to reproduce a task result. Two systems can load identical encoder weights and return different vectors or labels because they select another layer, use another valid-row mask, include prompt or special rows in a mean, apply a different projection, normalize at another point, or attach a label map in another order.

The release identity is therefore a tuple:

$$\mathcal{I}_{readout} = (A,T,X,H,R,L,S,U),$$

with:

A  backbone and task-head artifacts
T  tokenizer, special IDs, prompt/template, and truncation policy
X  sequence, segment, validity, content, and offset contract
H  contextual-state layer, axes, final norm, and dtype
R  row selection, pooling, projection, activation, and dropout
L  task type, ordered labels, null label, and span/tag policy
S  score type, normalization, threshold, and tie rule
U  runtime implementation, backend, export, and release identity

Changing one field creates a new semantic artifact even if every parameter shape remains loadable.

The manifest follows meaning rather than file layout. A model file, tokenizer folder, pooling configuration, and label map can be stored separately while still forming one indivisible release contract.

Manifest fields and their failure boundary

Record	Minimum fields	Shape-correct failure it prevents
artifact	backbone/head hashes, architecture, revision, adapter state	base encoder silently paired with another fine-tuned head
input	tokenizer hash, max length, truncation side, padding side, special IDs, prompt/template	first/last row points to a different semantic unit
row metadata	attention/content masks, polarity, segment IDs, offsets, word IDs	padding or special rows enter pooling; token labels lose source alignment
contextual state	layer or layer mixture, final norm, logical axes, parameter/compute/accumulation dtype	last layer is substituted for a trained layer mixture
readout	family, pooling mode, included rows, denominator, projection, activation, dropout	masked mean becomes padded mean or [CLS] selection
output identity	task type, ordered label map, null/no-answer ID, output dimension	label IDs are swapped under the same classifier shape
score	logit/probability/similarity type, vector normalization, temperature, threshold, tie rule	cosine threshold is applied to an inner-product model
runtime	framework and source revision, backend, export format, device/dtypes	exported graph omits mask or normalization behavior
release	schema, content ID, tests, parent release, migration and rollback compatibility	mutable files are promoted under an old model name

The manifest should carry both declared and observed runtime state. A requested backend or dtype is configuration; the backend and accumulation mode observed during a measured run are evidence.

Canonical manifest sketch

{
  "schema": "numinens.readout.v1",
  "artifact": {
    "backbone_sha256": "...",
    "head_sha256": "...",
    "architecture": "encoder",
    "task_revision": "..."
  },
  "input": {
    "tokenizer_sha256": "...",
    "max_length": 512,
    "padding_side": "right",
    "truncation": "longest_first",
    "special_token_ids": {"cls": 101, "sep": 102, "pad": 0},
    "prompt": null,
    "offset_mapping_required": true
  },
  "contextual_state": {
    "layer": -1,
    "axes": ["batch", "row", "channel"],
    "width": 768,
    "final_norm_applied": true,
    "compute_dtype": "float32",
    "accumulation_dtype": "float32"
  },
  "readout": {
    "family": "single_vector",
    "pooling": "masked_mean",
    "mask_source": "content_mask",
    "include_prompt": false,
    "include_special_tokens": false,
    "projection": null,
    "normalize": {"kind": "l2", "epsilon": 1e-12}
  },
  "output": {
    "kind": "embedding",
    "dimension": 768,
    "score_type": "cosine",
    "threshold": null
  },
  "runtime": {
    "framework_revision": "...",
    "observed_backend": "eager",
    "export_format": "safetensors+json"
  }
}

This is a semantic sketch, not a required universal serialization format. Official libraries may split the same identity among model config, tokenizer config, pooling modules, task-head files, and runtime arguments.

Derived invariants should not be typed twice

Some fields are consequences of others and should be derived by admission code:

contextual state width D
    -> pooling output width unless modes are concatenated
        -> projection input width
            -> projection output width
                -> embedding dimension or head input width

ordered label map
    -> class count C
        -> classifier output width

selected valid/content rows
    -> pooling denominator
        -> zero-row admission or fallback decision

For concatenated pooling modes, if mode $j$ emits width $D_j$, the projection input width is:

$$D_R=\sum_{j=1}^{J}D_j.$$

It must not be copied from a stale configuration after one mode is removed.

Admission pseudocode

def validate_readout_manifest(m):
    require(m.contextual_state.axes == ["batch", "row", "channel"])
    require(m.contextual_state.width > 0)
    require(m.input.padding_side in {"left", "right"})
    require(m.readout.mask_source in {"attention_mask", "content_mask"})

    if m.readout.pooling in {"masked_mean", "max", "weighted_mean"}:
        require(m.readout.mask_source is not None)

    derived_width = pooling_output_width(m.readout, m.contextual_state.width)
    if m.readout.projection is not None:
        require(m.readout.projection.in_features == derived_width)
        derived_width = m.readout.projection.out_features

    require(m.output.dimension == derived_width)

    if m.output.kind in {"classification", "token_classification"}:
        require(is_bijection(m.output.label2id, m.output.id2label))
        require(m.output.dimension == len(m.output.label2id))

    if m.output.score_type == "cosine":
        require(m.readout.normalize.kind == "l2" or
                m.output.cosine_normalizes_operands)

    require(m.runtime.framework_revision)
    require(m.release.content_id)

The executable fixture uses a five-row contextual tensor. Only two content rows enter the mean; [CLS], [SEP], and padding are excluded. Its expected normalized vector is derived from the selected rows, not copied into the manifest validator. Admission also fails closed when the content mask selects padding, because a content row must first be valid model input.

Manifest changes are migrations

Some changes preserve the backbone but still require a new readout release:

mean -> CLS pooling
include prompt -> exclude prompt
last layer -> learned layer mixture
dot product -> cosine with normalization
label order [negative, positive] -> [positive, negative]
max length 512 -> 256 with a different truncation policy

Migration must declare whether stored embeddings, cached document vectors, thresholds, calibration parameters, and evaluation baselines remain valid. For a bi-encoder, changing document pooling invalidates the document index even if query-side code is untouched. M3.11 owns that semantic invalidation; retrieval infrastructure owns rebuilding and serving the index.

0x02 A Contextual Row Has Several Identities

The encoder returns one vector per model row, but downstream code often asks about another unit: a source word, a character span, one member of an input pair, or ordinary content rather than inserted control state. These identities cannot be reconstructed from H.shape.

The source tokenizer article shows the broad path from text through normalization, pre-tokenization, subword modeling, post-processing, IDs, and embeddings:

Source-local A6 figure. It correctly exposes normalization, subword splitting, [CLS]/[SEP] insertion, integer IDs, and embedding lookup. For task readout, the pipeline must preserve additional row metadata: validity, special-row identity, source sequence, source offsets, and word ownership. M2.1 owns how that metadata is produced; M3.11 owns how a readout consumes it.

One row table, not one universal mask

Consider a paired input whose first source contains playing in Rome and whose second source contains who. A WordPiece-style trace may be:

row s	token	attention m	special k	sequence q	type ID	word ID	offset [a,b)
0	[CLS]	1	1	none	0	none	none
1	play	1	0	0	0	0	[0,4)
2	##ing	1	0	0	0	0	[4,7)
3	in	1	0	0	0	1	[8,10)
4	rome	1	0	0	0	2	[11,15)
5	[SEP]	1	1	none	0	none	none
6	who	1	0	1	1	0	[0,3)
7	[SEP]	1	1	none	1	none	none
8	[PAD]	0	1	none	0	none	none

The table distinguishes five predicates:

$$m_s^{valid}=\mathbf{1}[\text{row }s\text{ is model-visible}],$$ $$m_s^{special}=\mathbf{1}[\text{row }s\text{ was inserted as control state}],$$ $$m_s^{content}=m_s^{valid}(1-m_s^{special})m_s^{policy},$$ $$m_s^{(u)}=m_s^{valid}\mathbf{1}[q_s=u],$$ $$m_s^{source}=\mathbf{1}[o_s\ne\varnothing].$$

m_policy can exclude a visible prompt, instruction, title, or metadata field from final pooling while still allowing those rows to influence content through attention. This is why attention_mask cannot be renamed pooling_mask.

One contextual row can be model-valid but not poolable, source-mapped, or label-bearing. The diagram’s masks answer different questions and therefore must remain separate tensors or reproducibly derived views.

token_type_ids and sequence_ids are not synonyms

For BERT, token-type IDs index a learned segment embedding that is added before the encoder. They are model input. Tokenizer sequence_ids, by contrast, are alignment metadata that identify which source of a pair produced a row and normally use None for inserted special rows. Some model families omit learned token-type embeddings while fast tokenizers can still expose pair ownership.

Therefore this inference is invalid:

# Wrong as a general row-identity rule.
context_rows = token_type_ids == 1

A checkpoint may use all-zero type IDs, may not accept them, or may assign a type ID to [SEP] even though that row has no source offset. Use the tokenizer’s declared sequence map for source ownership; use token_type_ids only according to the checkpoint’s input contract. The official Tokenizers Encoding API separately exposes type_ids, sequence_ids, special_tokens_mask, word_ids, and offsets, which externally verifies this distinction.

Half-open offsets connect model rows back to source text

For a source string x, row s with offset

$$o_s=[a_s,b_s)$$

corresponds to x[a_s:b_s]. Half-open intervals make the token width $b_s-a_sand let adjacent pieces meet at one boundary without overlap. An inserted special token has no source substring; representing that fact asNoneis safer than treating(0,0)` as ordinary source content.

Given a character answer span $[a^*,b^*)$, a token row overlaps it when:

$$b_s>a^*\quad\land\quad a_s<b^*.$$

For extractive QA, overlap alone is not enough. The row must also belong to the declared context sequence. The official Transformers QA guide uses both offset mapping and sequence_ids for this reason. If truncation removes any part of the answer, the training policy must emit the declared no-answer/null target or drop the window; clipping the answer to the surviving edge creates a false label.

def rows_overlapping_span(offsets, sequence_ids, context_id, start, end):
    rows = []
    for s, (offset, sequence_id) in enumerate(zip(offsets, sequence_ids)):
        if sequence_id != context_id or offset is None:
            continue
        a, b = offset
        if b > start and a < end:       # [a,b) intersects [start,end)
            rows.append(s)
    return rows

A source word may own several model rows

In the trace, play and ##ing share word ID 0. A word-level label therefore needs an explicit expansion or reduction policy:

Policy	Training labels	Inference reduction	Consequence
first piece	supervise row 1; ignore row 2	read row 1	one loss term per source word
all pieces	copy or scheme-adjust label to rows 1 and 2	first, vote, or constrained decode	longer words receive more token-level terms unless reweighted
aggregate pieces	pool rows 1 and 2 before the word head	one word score	adds a word-readout operation

For first-piece supervision, with word and sequence identities (q_s,w_s), the label-bearing predicate is:

$$m_s^{first} = m_s^{content} \mathbf{1} \left[ (q_s,w_s)\ne(q_{s-1},w_{s-1}) \right].$$

Pair identity is included because each source sequence can restart word IDs at zero. The first content row in each sequence is also a first piece by definition. Loss-ignore values such as -100 are a supervision convention, not a substitute for preserving the mapping.

def first_piece_label_mask(attention, special, sequence_ids, word_ids):
    out = [False] * len(attention)
    previous = None
    for s in range(len(attention)):
        owner = (sequence_ids[s], word_ids[s])
        content = bool(attention[s]) and not bool(special[s])
        mapped = owner[0] is not None and owner[1] is not None
        out[s] = content and mapped and owner != previous
        if content and mapped:
            previous = owner
        else:
            previous = None
    return out

This reset-on-boundary implementation is deliberately explicit. A vectorized implementation is acceptable only if it preserves sequence boundaries, special rows, and left/right padding behavior.

First and last are semantic selectors, not physical columns

For row mask $m$, define:

$$s_{first}=\min\{s:m_s=1\}, \qquad s_{last}=\max\{s:m_s=1\}.$$

Then row selection is $r=H_{s_{first},:}$ or $r=H_{s_{last},:}$. The shortcut

last = attention_mask.sum(-1) - 1

works only for a right-padded contiguous visible prefix beginning at physical row zero. It fails for left padding, packed layouts, holes, and any selector whose mask excludes a visible suffix prompt. The robust operation gathers the minimum or maximum physical index satisfying the readout’s mask.

def boundary_index(mask, side):
    selected = [s for s, keep in enumerate(mask) if keep]
    if not selected:
        raise ValueError("readout selected zero rows")
    return min(selected) if side == "first" else max(selected)

Layer selection is part of row identity

The same token row has one state per encoder depth. A release must declare whether it consumes one layer,

$$H=H^{(\ell)},$$

an average over a fixed layer set,

$$H=\frac{1}{|\mathcal L|}\sum_{\ell\in\mathcal L}H^{(\ell)},$$

or a learned scalar mixture,

$$\alpha_\ell = \frac{e^{a_\ell}}{\sum_{j\in\mathcal L}e^{a_j}}, \qquad H=\gamma\sum_{\ell\in\mathcal L}\alpha_\ell H^{(\ell)}.$$

The learned logits $a_\ell$ and scale $\gamma$ are task-head parameters and must travel with the readout checkpoint. Replacing a mixture with the last hidden state preserves [B,S,D] while changing every downstream row.

Source-to-tensor admission trace

encoding = tokenizer(
    first_text,
    second_text,
    return_offsets_mapping=True,
    return_special_tokens_mask=True,
    truncation="longest_first",
    padding="max_length",
)

row_contract = {
    "input_ids": encoding["input_ids"],
    "attention_mask": encoding["attention_mask"],
    "special_tokens_mask": encoding["special_tokens_mask"],
    "sequence_ids": encoding.sequence_ids(),
    "word_ids": encoding.word_ids(),
    "offsets": normalize_absent_offsets(encoding["offset_mapping"]),
}
validate_row_contract(row_contract)

H = model(**model_inputs(row_contract), output_hidden_states=True).hidden_states
contextual = select_declared_layer_or_mixture(H, manifest.contextual_state)
content_mask = derive_content_mask(row_contract, manifest.readout)

Fast-tokenizer alignment APIs are an implementation boundary, not a universal property of every tokenizer backend. If a runtime cannot emit the metadata a task requires, admission must fail or preprocessing must persist an equivalent mapping. It must not guess offsets from decoded token strings after normalization.

Zero-row and empty-source policy

Every reduction needs a declared empty-set behavior:

reject:
    safest when an empty content set means preprocessing drift

use a learned empty vector:
    valid only if that vector was trained and versioned

fall back to a declared special row:
    valid only if the training/readout contract used that rule

clamp denominator and return zero:
    numerically finite, but semantically dangerous if accidental

The default executable oracle rejects zero selected rows. Section 0x03 derives why a denominator clamp prevents NaN but does not decide the semantic policy.

0x03 Pooling Is A Reduction With A Backward Contract

Let one example expose contextual rows

$$H\in\mathbb{R}^{S\times D}$$

and a declared readout mask $m\in\{0,1\}^{S}$. Write

$$\mathcal V=\{s:m_s=1\},\qquad n=|\mathcal V|=\sum_s m_s.$$

Every pool below changes how information and gradient cross the row axis. The choice is therefore learned architecture or immutable configuration, not a cosmetic post-processing flag.

Selection routes state through one row. Fixed reductions distribute state by a declared rule. Learned pooling makes the weighting function part of the checkpoint. Concatenating modes preserves several summaries and widens the next projection.

First and last valid row: gather, do not reduce

With the semantic selectors from 0x02, first-row pooling is:

$$r_d=H_{s_{first},d}.$$

If $g_d=\partial\mathcal L/\partial r_d$, then:

$$\frac{\partial\mathcal L}{\partial H_{s,d}} = \mathbf{1}[s=s_{first}]g_d.$$

Last-row pooling substitutes $s_{last}$. No gradient reaches another output row through the gather itself, although contextualization may already have mixed information into the selected row inside the backbone.

This distinction matters for BERT terminology. The official BERT implementation does not return the raw first row as pooler_output; BertPooler computes:

$$p=\tanh(W_pH_{:,0,:}+b_p).$$

The physical first row is appropriate only because the BERT input grammar puts [CLS] there. Raw [CLS], transformed BERT pooler output, and “first non-pad row” are three different readouts. A checkpoint trained with one cannot silently substitute another.

Masked mean: derive the denominator

The arithmetic mean over selected rows is:

$$r_d = \frac{1}{n}\sum_{s=1}^{S}m_sH_{s,d}.$$

The denominator is $n$, not padded width $S$. Padding one example from 8 to 512 physical rows must not shrink its representation by $8/512$.

Because the binary mask is metadata rather than differentiable state, the local Jacobian is:

$$\frac{\partial r_d}{\partial H_{s,j}} = \frac{m_s}{n}\mathbf{1}[d=j].$$

For upstream gradient $g$:

$$\frac{\partial\mathcal L}{\partial H_{s,:}} = \frac{m_s}{n}g.$$

Every selected row receives an equal direct share. That does not mean every source token has equal total influence: contextualization, subword expansion, layer selection, and later normalization still matter.

def masked_mean(hidden, mask):
    # hidden [B,S,D], mask [B,S], 1 means this readout includes the row
    x = hidden.float()                         # accumulator contract
    w = mask.to(dtype=x.dtype).unsqueeze(-1)  # [B,S,1]
    count = w.sum(dim=1)                       # [B,1]
    if torch.any(count == 0):
        raise ValueError("masked mean selected zero rows")
    return (x * w).sum(dim=1) / count          # [B,D]

Promoting the reduction to FP32 reduces long-sum error for lower-precision backbone outputs. Returning FP16/BF16 afterward is a separate output-dtype decision. The denominator and numerator should be accumulated under the same declared precision contract.

Mean divided by square-root length is not a mean

Some released pooling modules expose:

$$r^{sqrt}_d = \frac{\sum_s m_sH_{s,d}}{\sqrt n}.$$

Its backward share is $m_sg/\sqrt n$. If selected rows have a common mean $\mu$, then $r^{sqrt}=\sqrt n\,\mu$; representation magnitude can therefore grow with sequence length. Later L2 normalization removes global scale from cosine geometry but not from pre-normalization nonlinear projections, dot products, or gradient magnitudes. The name must not be shortened to “mean pooling.”

Masked max: zero is not a safe mask value

Channel-wise max pooling computes:

$$r_d=\max_{s\in\mathcal V}H_{s,d}.$$

This implementation is wrong:

maximum = (hidden * mask.unsqueeze(-1)).max(dim=1).values

If every selected value in one channel is negative, a masked pad becomes zero and wins. Invalid rows must be replaced before the reduction by an identity sentinel such as $-\infty$ in a supported accumulation dtype:

def masked_max(hidden, mask):
    if torch.any(mask.sum(dim=1) == 0):
        raise ValueError("masked max selected zero rows")
    masked = hidden.float().masked_fill(~mask.bool().unsqueeze(-1), -torch.inf)
    values, indices = masked.max(dim=1)
    return values, indices

When one row uniquely maximizes channel $d$, direct gradient for that channel goes only to that row. At ties, max has a set-valued mathematical subgradient; libraries choose a concrete backward convention. Returning/storing argmax indices makes that convention auditable. Max can select different rows for different channels, so the output need not equal any one token vector.

Fixed weighted means expose position or policy bias

For declared nonnegative weights $w_s$:

$$Z=\sum_s m_sw_s, \qquad r=\frac{1}{Z}\sum_s m_sw_sH_s.$$

The direct hidden-state gradient is:

$$\frac{\partial\mathcal L}{\partial H_s} = \frac{m_sw_s}{Z}g.$$

Weights can encode recency, position, confidence, or source-specific policy. They remain configuration only if they are deterministic functions recorded in the manifest. For example, position-weighted mean pooling is not the same as a learned importance model merely because both emit [B,D].

Negative weights are mathematically possible but change the denominator and can make $Z$ zero through cancellation. A “weighted mean” contract should state whether weights are nonnegative, whether they are normalized, and what happens at zero total mass.

Learned scalar attention adds an indirect gradient path

A common learned pool scores every selected row:

$$u_s=v^\top\tanh(WH_s+b),$$

masks invalid logits, normalizes over rows,

$$\alpha_s = \frac{m_se^{u_s}}{\sum_jm_je^{u_j}},$$

and returns:

$$r=\sum_s\alpha_sH_s.$$

For upstream vector $g$, softmax coupling gives:

$$\frac{\partial\mathcal L}{\partial u_s} = \alpha_sg^\top(H_s-r).$$

The hidden row receives a direct value path and an indirect score path:

$$\frac{\partial\mathcal L}{\partial H_s} = \alpha_sg + \alpha_sg^\top(H_s-r) W^\top\left[v\odot\left(1-\tanh^2(WH_s+b)\right)\right].$$

The second term is precisely what a fixed weighted mean lacks. Changing W, v, mask polarity, score dtype, or softmax axis changes the learned readout.

def attention_pool(hidden, mask, scorer):
    score = scorer(hidden.float()).squeeze(-1)          # [B,S]
    score = score.masked_fill(~mask.bool(), -torch.inf)
    if torch.any(mask.sum(dim=-1) == 0):
        raise ValueError("attention pool selected zero rows")
    alpha = torch.softmax(score, dim=-1)                # row axis only
    pooled = torch.einsum("bs,bsd->bd", alpha, hidden.float())
    return pooled, alpha

Structured self-attentive pooling generalizes one scalar distribution to $R$ attention hops. With rows arranged as $H\in\mathbb R^{S\times D}$:

$$A = \operatorname{softmax}_{S} \left(W_2\tanh(W_1H^\top)\right) \in\mathbb R^{R\times S},$$ $$M=AH\in\mathbb R^{R\times D}.$$

Lin et al. add a diversity penalty based on $AA^\top$ to discourage identical hops. This is an externally verified paper design, not a requirement for every attention pool. The method emits a matrix until a later flatten/reduction step; calling it a single vector hides an output-shape decision.

Combining pooling modes widens the artifact

Sentence Transformers exposes modular pooling and can concatenate enabled modes. For $J$ readouts $r^{(j)}\in\mathbb R^{D_j}$:

$$r^{cat}=[r^{(1)};\ldots;r^{(J)}] \in\mathbb R^{\sum_jD_j}.$$

If all preserve hidden width, the next layer sees $JD$, not $D$. Concatenating raw first-row, mean, and max summaries therefore triples projection input width and parameter count. Averaging those summaries would keep width $D$ but define a different architecture.

Alternatives by contract

Readout	Trainable state	Direct row gradient	Main sensitivity
first/last gather	none, unless followed by pooler	one selected row	grammar, padding, suffix/prompt
BERT pooler	W_p,b_p	first row through tanh	checkpoint pooler training
masked mean	none	equal selected-row shares	row set and denominator
mean/sqrt-length	none	equal 1/sqrt(n) shares	length-dependent norm
channel max	none	per-channel argmax rows	outliers and tie convention
fixed weighted mean	configured weights	normalized weight shares	position/policy bias
learned attention	scorer parameters	value plus score path	score saturation and collapse
multi-hop attention	scorer plus later reducer	multiple distributions	hop redundancy and output shape
concatenated modes	constituent state	union of constituent paths	widened projection/interface

No row-only pooling universally repairs a backbone that was not trained for the target representation geometry. SBERT’s siamese/triplet training, SimCSE’s contrastive objective, and task-specific fine-tuning change what the rows encode; they are not merely better pooling formulas. Section 0x07 owns their representation-objective contracts and readout-local trainable-state schedules.

Numerical and semantic assertions

assert hidden.ndim == 3 and mask.ndim == 2
assert hidden.shape[:2] == mask.shape
assert mask.dtype == torch.bool
assert torch.all(mask.any(dim=1))
assert accumulation_dtype in {torch.float32, torch.float64}

pooled = pool(hidden, mask)
assert pooled.shape == (hidden.shape[0], declared_pool_width)
assert torch.isfinite(pooled).all()

clamp_min(epsilon) is useful inside a numerically defensive implementation, but assertions must run before the clamp if zero selected rows are forbidden. Otherwise an invalid example becomes a finite zero vector and can enter an ANN index or classifier without an obvious failure.

0x04 Task Heads Type The Output Axis

Pooling answers “which contextual state summarizes this input?” A task head answers another question: “which target axis is represented by each output coordinate?” A [B,3] tensor can mean three mutually exclusive classes, three independent attributes, three regression targets, or three candidate scores. Shape alone does not choose among them.

A pooled branch removes the row axis before prediction. Token and span branches preserve it. Each branch carries a distinct target identity, loss reduction, decision rule, and release state.

Single-label sequence classification

Let a readout produce $r_b\in\mathbb R^{D_r}$ and let the ordered label map have $C$ entries. A linear head computes:

$$z_b=W_cr_b+b_c, \qquad W_c\in\mathbb R^{C\times D_r}, \qquad z_b\in\mathbb R^C.$$

The head adds $CD_r+C$ parameters. For one mutually exclusive target $y_b$:

$$p_{b,c} = \frac{e^{z_{b,c}}}{\sum_{j=1}^{C}e^{z_{b,j}}}, \qquad \mathcal L_b=-\log p_{b,y_b}.$$

The familiar softmax-cross-entropy gradient follows from the shared denominator:

$$\frac{\partial\mathcal L_b}{\partial z_{b,c}} = p_{b,c}-\mathbf 1[c=y_b].$$

The gradient handed to the readout is:

$$\frac{\partial\mathcal L_b}{\partial r_b} = W_c^\top(p_b-e_{y_b}).$$

Thus changing label order while retaining shape $C$ silently permutes rows of W_c, probabilities, metrics, and decisions. label2id and id2label are checkpoint identity, not display metadata.

def sequence_classification(hidden, row_mask, pool, head, labels=None):
    r = pool(hidden, row_mask)        # [B,D_r]
    r_drop = dropout(r)               # active only in training mode
    logits = head(r_drop)             # [B,C]
    loss = None if labels is None else cross_entropy(logits, labels)
    return {"logits": logits, "loss": loss, "readout": r}

Dropout placement and probability are part of the trained head. Evaluation must switch dropout off; “same weights” in training mode is not deterministic inference.

Regression keeps coordinate meaning outside softmax

For $R$ real-valued targets:

$$\hat y_b=W_rr_b+b_r\in\mathbb R^R.$$

With mean squared error over $BR$ values:

$$\mathcal L_{MSE} = \frac{1}{BR}\sum_{b=1}^{B}\sum_{j=1}^{R} (\hat y_{b,j}-y_{b,j})^2,$$ $$\frac{\partial\mathcal L_{MSE}}{\partial\hat y_{b,j}} = \frac{2}{BR}(\hat y_{b,j}-y_{b,j}).$$

No softmax couples regression coordinates. Target scaling therefore changes gradient scale. A release must preserve target units, normalization statistics, inverse transform, missing-target mask, and whether MSE, MAE, Huber, Gaussian negative log-likelihood, or another objective trained the head.

Huber loss is a useful alternative when large residuals should become linear:

$$\ell_\delta(e)= \begin{cases} \frac12e^2,&|e|\le\delta,\\ \delta(|e|-\frac12\delta),&|e|>\delta. \end{cases}$$

The threshold $\delta$ is part of the objective contract, not an inference threshold.

Multilabel classification uses independent Bernoulli coordinates

For target vector $y_b\in\{0,1\}^C$, every coordinate can be positive:

$$p_{b,c}=\sigma(z_{b,c}),$$ $$\mathcal L_b = -\sum_{c=1}^{C} \left[ y_{b,c}\log\sigma(z_{b,c}) + (1-y_{b,c})\log(1-\sigma(z_{b,c})) \right].$$

For an unweighted sum, the logit gradient is:

$$\frac{\partial\mathcal L_b}{\partial z_{b,c}} = \sigma(z_{b,c})-y_{b,c}.$$

This is not one softmax over the $C$ labels. Probabilities do not sum to one, and inference may emit zero, one, or many labels. Per-label thresholds, positive-class weights, focal terms, and calibration parameters must be versioned with the ordered label map. A universal 0.5 threshold is a policy choice, not a theorem.

Multiple choice moves the competition to the candidate axis

For $K$ answer candidates, each candidate is rendered as its own paired input:

input IDs       [B,K,S]
backbone rows   [B,K,S,D]
pooled states   [B,K,D_r]
scalar head     [B,K,1]
candidate logits [B,K]

Flattening B*K for the backbone is an implementation schedule; unflattening before candidate softmax restores the semantic axis:

$$z_{b,k}=w^\top r_{b,k}+b, \qquad p_{b,k}=\operatorname{softmax}_{k}(z_b).$$

Candidates must share one example identity and deterministic order. Treating all flattened rows as independent binary examples changes the objective.

Token classification preserves the model-row axis

For $H\in\mathbb R^{B\times S\times D}$:

$$Z_{b,s,:}=W_tH_{b,s,:}+b_t \in\mathbb R^C.$$

Let $\lambda_{b,s}=1$ only for rows that bear a target under the first-piece, all-piece, or aggregate policy from 0x02. With $N_L=\sum_{b,s}\lambda_{b,s}$:

$$\mathcal L_{token} = -\frac{1}{N_L} \sum_{b,s}\lambda_{b,s} \log p_{b,s,y_{b,s}}.$$

The local logit gradient is:

$$\frac{\partial\mathcal L_{token}} {\partial Z_{b,s,c}} = \frac{\lambda_{b,s}}{N_L} \left(p_{b,s,c}-\mathbf 1[c=y_{b,s}]\right).$$

Padding, special tokens, ignored subword pieces, and truncated source words must contribute zero. The predicted row labels still need offsets/word IDs to return source spans; decoding token strings and searching them in normalized text is not a reversible alignment algorithm.

logits = token_head(dropout(H))                    # [B,S,C]
flat_loss = cross_entropy(
    logits.reshape(-1, C),
    aligned_labels.reshape(-1),
    ignore_index=IGNORE,
    reduction="none",
).reshape(B, S)
loss = flat_loss[label_mask].mean()

A linear-chain CRF is an alternative when adjacent labels obey a learned transition model. It adds transition scores $A_{i,j}$, normalizes over complete label sequences, and uses Viterbi or constrained decoding. BIO/BILOU legality can instead be imposed only at inference. These designs have different learned state and losses; “token classification” does not imply either one.

Extractive QA predicts boundaries over context rows

BERT-style extractive QA applies one linear map with two output channels:

$$[z^{start}_{b,s},z^{end}_{b,s}] = W_qH_{b,s,:}+b_q, \qquad W_q\in\mathbb R^{2\times D}.$$

This is equivalent to separate start/end vectors only at the linear map; the two channels still have distinct parameters. A common training objective is:

$$\mathcal L_{span} = \frac12 \left[ -\log\operatorname{softmax}(z^{start})_{s^*} -\log\operatorname{softmax}(z^{end})_{e^*} \right],$$

after masking rows that cannot be answers. Start and end are trained with separate normalized distributions, so the loss does not itself guarantee $s\le e$ or a maximum answer length.

Inference must search the admissible span set:

$$\mathcal A = \{(s,e): m_s^{context}=m_e^{context}=1, s\le e, e-s+1\le L_{answer}\}.$$

For independent boundary logits, a standard score is:

$$(\hat s,\hat e) = \arg\max_{(s,e)\in\mathcal A} \left(z^{start}_s+z^{end}_e\right).$$

The selected offsets map rows back to a source substring. This constrained joint search is not equivalent to independently taking argmax(start) and argmax(end), which can return an inverted or overlong span.

def best_valid_span(start, end, context_mask, max_answer_length):
    best = None
    for s in range(len(start)):
        if not context_mask[s]:
            continue
        stop = min(len(end), s + max_answer_length)
        for e in range(s, stop):
            if not context_mask[e]:
                continue
            candidate = (start[s] + end[e], s, e)
            if best is None or candidate > best:
                best = candidate
    if best is None:
        raise ValueError("no admissible answer span")
    return best[1], best[2], best[0]

No-answer support adds another contract. One design uses a declared null row such as [CLS] and compares its boundary score with the best non-null span; another adds a separate answerability classifier. The threshold is calibrated on a development distribution and must be released. SQuAD 1.1’s original task contains answer spans, while no-answer handling is associated with later task variants and model recipes; do not project one convention onto every extractive QA checkpoint.

Joint span scorers, biaffine boundary models, and span-enumeration classifiers can represent start/end interaction directly but cost more than two independent row channels. They are alternatives, not implementation details of the BERT head.

Head, objective, and decision ledger

Task	Head output	Training coupling	Inference decision
multiclass	[B,C] logits	softmax across labels	argmax/top-k plus calibration policy
regression	[B,R] values/parameters	coordinate loss or probabilistic NLL	inverse transform and domain constraints
multilabel	[B,C] logits	independent/weighted sigmoid terms	per-label thresholds
multiple choice	[B,K] logits	softmax across candidates	candidate argmax
token labels	[B,S,C] logits	masked row losses or structured sequence loss	offset-aware merge/constrained decode
extractive QA	two [B,S] score fields	independent or joint boundary objective	constrained span plus no-answer policy

Fail-closed head admission

def validate_task_head(manifest, batch, output):
    require(output.shape == manifest.output.logical_shape(batch))
    require(manifest.output.dimension == manifest.head.out_features)
    require(manifest.output.ordered_labels_are_bijective())

    if manifest.output.kind == "token_classification":
        require(batch.label_mask.shape == batch.input_ids.shape)
        require(batch.offsets_available_for_emitted_rows())

    if manifest.output.kind == "extractive_qa":
        require(batch.context_mask.shape == batch.input_ids.shape)
        require(batch.context_offsets_are_present())
        require(manifest.output.max_answer_length > 0)

    if manifest.output.kind == "multilabel":
        require(len(manifest.output.thresholds) == manifest.output.dimension)

The executable oracle checks local losses and gradients, source-row masking, candidate-axis competition, and constrained span search. M3.9 remains the owner of optimizer state, accumulation, distributed reduction, and checkpoint timing.

0x05 A Score Is Not Yet A Decision

M2.2 ended at model-input state; this article therefore owns the downstream representation geometry. This section derives normalization and score meaning. Section 0x07 owns the objectives that train that geometry. Physical ANN realization remains a retrieval-system concern. The local endpoint here is a typed, calibrated, and releasable decision.

Normalize only under the trained geometry

For nonzero vector $x\in\mathbb R^D$:

$$\hat x=\frac{x}{\lVert x\rVert_2}.$$

The local Jacobian is:

$$\frac{\partial\hat x}{\partial x} = \frac{1}{\lVert x\rVert_2} \left(I-\hat x\hat x^\top\right).$$

The projection removes radial gradient. Applying L2 normalization only at deployment is therefore not equivalent to training through it. Near-zero norms also amplify the Jacobian; an epsilon or zero-vector fallback changes the function and must be declared.

Three common pair scores are:

$$s_{dot}(q,d)=q^\top d,$$ $$s_{cos}(q,d)= \frac{q^\top d}{\lVert q\rVert_2\lVert d\rVert_2},$$ $$\delta_{L2}^2(q,d)=\lVert q-d\rVert_2^2.$$

For unit vectors:

$$\delta_{L2}^2(\hat q,\hat d)=2-2\hat q^\top\hat d.$$

Thus maximum cosine, maximum inner product over consistently unit-normalized vectors, and minimum squared L2 have the same exact ranking. They do not emit the same numbers. A threshold cosine >= 0.8 cannot be copied to squared L2 as distance <= 0.8; under the identity it becomes distance <= 0.4.

Raw inner product is different when norms vary:

$$q^\top d = \lVert q\rVert_2\lVert d\rVert_2\cos\theta.$$

Norm can encode learned confidence, popularity, frequency, or accidental scale. Removing it is a model change, not automatically a correction.

Score types form a small type system

Type	Range	Larger wins?	Sum constraint	Probability claim
raw class logit	real	only relative within declared class head	none	no
softmax probability	[0,1]	yes	one over class/candidate axis	conditional model probability, not necessarily calibrated
sigmoid probability	[0,1]	yes	none across labels	marginal model probability, not necessarily calibrated
dot/cosine similarity	real / [-1,1]	yes	none	no
L2 distance	nonnegative	no	none	no
calibrated probability	[0,1]	yes	depends on calibrator/task	empirical claim on a declared population

Calling every float score throws away exactly the information a decision needs. APIs and manifests should use names such as logits, probabilities, cosine_similarity, or squared_l2_distance.

Ranking, calibration, and action policy are separate stages. A monotone calibrator can preserve ranking while changing thresholds and probability meaning; a changed encoder can invalidate all three.

Temperature changes confidence even when argmax is fixed

Given class logits $z$ and temperature $T>0$:

$$p_c(T)= \frac{\exp(z_c/T)}{\sum_j\exp(z_j/T)}.$$

Positive scaling preserves argmax(z), but changes entropy, NLL, and any probability threshold. For target $y$:

$$\frac{\partial[-\log p_y(T)]}{\partial z_c} = \frac{1}{T} \left(p_c(T)-\mathbf 1[c=y]\right).$$

Training temperature in a contrastive loss controls optimization and negative competition; post-hoc classification temperature scaling fits $T$ on held-out logits to improve calibration. The two parameters may use the same equation but belong to different lifecycle stages.

Temperature scaling cannot repair class-conditional errors that require different transformations. Alternatives include vector/matrix scaling, Platt sigmoid calibration for a scalar score, isotonic regression, or a separate calibration model. Greater flexibility raises overfitting risk and expands release state.

Calibration is population-conditional

A predictor is calibrated on population $P$ when, informally:

$$\Pr(Y=1\mid\hat p=p)=p.$$

Finite data requires estimates. Negative log-likelihood and Brier score are proper scoring rules; reliability diagrams and expected calibration error bin predictions and therefore depend on binning policy. Accuracy alone cannot detect a model that is systematically overconfident.

def fit_temperature(frozen_logits, labels, calibration_split):
    assert calibration_split.is_disjoint_from_train_and_test
    log_T = Parameter(0.0)
    optimize(lambda: cross_entropy(frozen_logits / exp(log_T), labels))
    return exp(log_T).detach()

The calibrator identity includes:

base checkpoint and readout release
calibration data revision and inclusion rules
label order and population slice
calibrator family, parameters, and fitting objective
software/dtype used to produce calibration logits

Domain, prevalence, class mix, and input-policy shifts can break calibration while preserving ranking or aggregate accuracy. Monitor calibration on the deployment population rather than treating it as a permanent model property.

Thresholds encode costs and capacity

For calibrated binary probability $p=\Pr(Y=1\mid x)$, false-positive cost $C_{FP}$, and false-negative cost $C_{FN}$:

predict positive: expected cost = (1-p) * C_FP
predict negative: expected cost = p * C_FN

Choose positive when:

$$p\ge \frac{C_{FP}}{C_{FP}+C_{FN}}.$$

The familiar 0.5 follows only when the two costs are equal. Capacity limits, review queues, prevalence shift, legal constraints, and fairness requirements can require another policy. For multilabel output, each label can have its own threshold and cost model.

Similarity thresholds need empirical relevance labels. A cosine value is not a probability, and its numeric scale can shift after changing pooling, prompts, normalization, quantization, or the document population. Top-k retrieval avoids one absolute cutoff but introduces a capacity choice and can still return irrelevant items when all scores are poor.

Abstention adds another decision region

One simple multiclass policy is:

$$\hat y= \begin{cases} \arg\max_c p_c,&\max_c p_c\ge t_{accept},\\ \text{abstain},&\text{otherwise}. \end{cases}$$

Margin, entropy, conformal sets, retrieval-score gaps, or an explicit reject class are alternatives. They make different statistical promises. An abstention threshold must be evaluated with coverage and conditional risk, not accuracy on accepted examples alone.

Monotone transforms preserve rankings, not decisions

For strictly increasing $f$:

$$s_i>s_j\iff f(s_i)>f(s_j).$$

Exact top-k order is preserved, but these objects can change:

absolute threshold crossings
score margins and gap alarms
probability calibration
weighted fusion with another score
floating-point tie behavior
approximate-index traversal

Platt scaling with positive slope can preserve one score’s order while giving a probability interpretation on calibration data. A negative fitted slope reverses the order and usually signals that score orientation was misdeclared or the relationship differs from expectation.

Readout decision admission

def decide(output, contract):
    require(output.score_type == contract.expected_score_type)

    if contract.metric == "cosine":
        require(output.query_normalized and output.document_normalized)
    if contract.calibrator is not None:
        require(contract.calibrator.parent_release == output.release_id)
        score = contract.calibrator(output.score)
    else:
        score = output.score

    require(contract.threshold_revision)
    require(contract.tie_rule)
    return apply_threshold_abstention_and_capacity(score, contract)

Changing the backbone, tokenizer, selected layer, pool, projection, normalization, score metric, or output dtype invalidates calibration and thresholds unless a frozen equivalence test proves otherwise. It may also invalidate every stored document vector. M3.11 owns that semantic invalidation declaration; the retrieval-system owner executes re-encoding and index rebuild, and M8.10 owns production transition.

0x06 Interaction Placement Determines Reusable State

Cross-encoders, bi-encoders, and late-interaction models all emit pairwise scores, but they do not compute the same intermediate object. The architectural question is:

At what point may query state and document state interact?

Moving that point changes representation semantics, offline artifacts, online work, and the gradients available during training. It is not merely a faster implementation of one equation.

The reusable document artifact grows from none, to one vector, to many token vectors. In return, the interaction retained at query time grows from one global metric to token-level matching; only the cross-encoder permits full query-document attention inside every Transformer layer.

Cross-encoder: interaction before representation readout

A BERT-style pair grammar is:

[CLS] query [SEP] document [SEP]

The tokenizer emits one pair sequence $x_{q,d}$. Full-visibility encoder layers allow every admitted query row to affect every admitted document row:

$$H_{q,d}=F_\theta(x_{q,d},m_{q,d}) \in\mathbb R^{S_{pair}\times D}.$$

A pair readout and scalar or class head follow:

$$s_{cross}(q,d)=w^\top R(H_{q,d})+b.$$

The output may be an uncalibrated logit, regression value, or class vector. Official Sentence Transformers documentation explicitly notes that cross-encoder checkpoints may return logits and that an optional sigmoid changes score scale without changing rank. The release must declare the actual type.

Document state cannot generally be precomputed independently because:

$$F_\theta([q;d])_d \ne F_\theta(d)$$

when document rows attend to query rows. Caching tokenized document text is possible; caching its joint contextual state for arbitrary future queries is not the same computation.

Pair rendering also creates a shared length budget. A max_length=512 contract must declare whether the query or document truncates first, the stride/window policy for long documents, and how chunk scores aggregate. Query-dependent truncation can change which evidence the scorer sees.

def cross_score(query, documents, tokenizer, model):
    pairs = [(query, document) for document in documents]
    batch = tokenizer(
        pairs,
        padding=True,
        truncation="only_second",
        max_length=PAIR_LIMIT,
    )
    logits = model(**batch).logits
    return type_scores(logits, SCORE_CONTRACT)

Bi-encoder: interaction after one-vector readout

A bi-encoder computes each side independently:

$$u_q=R_q(F_{\theta_q}(T_q(q)))\in\mathbb R^{D_e},$$ $$v_d=R_d(F_{\theta_d}(T_d(d)))\in\mathbb R^{D_e},$$

then applies a decomposable score:

$$s_{bi}(q,d)=u_q^\top v_d$$

or the declared cosine/L2 alternative. For query matrix $U\in\mathbb R^{B_q\times D_e}$ and document matrix $V\in\mathbb R^{B_d\times D_e}$, exact inner-product scores are:

$$S=UV^\top\in\mathbb R^{B_q\times B_d}.$$

Because $v_d$ is query-independent, document vectors are reusable. That is the architectural property that permits an offline document artifact and exact or approximate vector search. M3.11 owns the vector and score semantics; retrieval systems own index construction and search execution.

The encoders may share all weights, start equal and later diverge, or be different architectures. DPR’s primary paper uses separate question and passage BERT encoders and an inner-product score. A shared-weight Sentence-BERT path is a different parameter-ownership contract even when both are called bi-encoders.

Asymmetric query/document templates are also state:

query:    "query: {text}"
document: "passage: {title} {body}"

Swapping templates or encoders can preserve vector width while moving the two sides into incompatible spaces.

Late interaction: retain one vector per selected row

ColBERT moves interaction later than a cross-encoder but earlier than a single-vector bi-encoder. Independent encoders and a projection produce:

$$Q=[q_1,\ldots,q_m]^\top\in\mathbb R^{m\times d},$$ $$D=[d_1,\ldots,d_n]^\top\in\mathbb R^{n\times d},$$

with selected query/document rows and, in the paper contract, normalized token vectors. The pairwise token matrix is:

$$A=QD^\top\in\mathbb R^{m\times n}.$$

ColBERT’s late-interaction score is MaxSim followed by a query-token sum:

$$s_{late}(q,d) = \sum_{i=1}^{m} \max_{1\le j\le n}q_i^\top d_j.$$

Each query token independently retrieves its strongest document-token match. The sum retains more local evidence than reducing the document to one vector, but document storage now retains up to $n$ projected vectors plus row metadata. Physical byte, compression, and kernel accounting belong to M4/M8.

Padding or excluded document rows must be masked with $-\infty$ before each row-wise max. Multiplying an invalid dot product by zero repeats the masked-max failure from pooling.

def maxsim(query_tokens, document_tokens, query_mask, document_mask):
    # Q [B,m,d], D [B,n,d]
    similarity = query_tokens.float() @ document_tokens.float().transpose(-1, -2)
    similarity = similarity.masked_fill(~document_mask[:, None, :], -torch.inf)
    per_query_token, winner = similarity.max(dim=-1)   # [B,m]
    per_query_token = per_query_token.masked_fill(~query_mask, 0.0)
    return per_query_token.sum(dim=-1), winner

For a unique winning document row

$$j^*(i)=\arg\max_jq_i^\top d_j,$$

the local score gradients are:

$$\frac{\partial s_{late}}{\partial q_i}=d_{j^*(i)},$$ $$\frac{\partial s_{late}}{\partial d_j} = \sum_{i:j^*(i)=j}q_i.$$

Several query rows can route gradient to one document row. At ties, the concrete argmax convention matters. Smooth alternatives such as log-sum-exp distribute mass across document rows but define a different scorer and temperature:

$$\operatorname{LSE}_\tau(a_1,\ldots,a_n) = \tau\log\sum_j e^{a_j/\tau}.$$

As $\tau\to0^+$ it approaches max. For finite $\tau$, it is not MaxSim and can change ranking.

Paper-by-contract comparison

Contract	Interaction point	Reusable document state	Pair score state	Main semantic loss if substituted
cross-encoder	inside every joint Transformer layer	no query-independent contextual state	joint rows plus task head	removes token-level cross-input attention
tied bi-encoder	after independent pooled readouts	one vector	shared encoder/pool plus metric	forces one representation function on both sides
untied bi-encoder / DPR-style	after independent pooled readouts	one vector	separate query/document encoders plus metric	swapping sides or checkpoints breaks space alignment
ColBERT-style late interaction	after independent token projections	many document row vectors	projection, row masks, normalization, MaxSim	one-vector pooling removes local matching

Cross-encoder source-reported quality advantages do not establish a universal ordering: candidate recall, pair truncation, training data, latency budget, and task distribution all matter. The architectures expose different feasible workflows rather than one Pareto ranking.

A two-stage system preserves both ceilings

A common design retrieves $k_0$ candidates with a reusable bi-encoder or late interaction artifact, then applies a cross-encoder to those pairs. The reranker cannot recover a relevant document absent from its input set:

$$\operatorname{Recall}_{final@k} \le \operatorname{Recall}_{candidate@k_0}.$$

M3.11 owns the semantic candidate/reranker interface. Retrieval systems own the index-specific serving manifest:

candidate stage output:
    stable document IDs + retrieval score type + candidate rank

reranker input:
    exact query/document rendering + parent candidate identity

reranker output:
    typed pair logit/probability + stable tie rule

Interaction-contract admission

def validate_interaction_release(m):
    if m.kind == "cross_encoder":
        require(not m.document_state_reusable)
        require(m.pair_template_hash and m.pair_truncation_policy)
    elif m.kind == "bi_encoder":
        require(m.document_vector_dimension == m.query_vector_dimension)
        require(m.query_release and m.document_release)
        require(m.metric_contract)
    elif m.kind == "late_interaction":
        require(m.query_row_policy and m.document_row_policy)
        require(m.projection_revision and m.maxsim_tie_rule)
        require(m.document_token_artifact_schema)
    else:
        raise ValueError("unsupported interaction contract")

All three paths still need task-specific training. Section 0x07 owns the local objective, contrastive denominator, and backward-state contract for these readouts.

Representation families change more than one pooling switch

The preceding sections deliberately define reusable primitives: visibility, row identity, selection, pooling, projection, normalization, interaction, and score type. A named embedding model composes several of them. Replacing only the pool while keeping the rest of the pipeline fixed usually does not reproduce the named method.

A useful model-side identity is:

(backbone checkpoint, visibility policy, rendered input, selected layer,
 selected rows, pool/projection, normalization, objective stages,
 positive/negative relation, added trainable state, exported state)

This subsection keeps the method comparison at M3.11’s owner boundary. It does not repeat the history of static and contextual word representations, and it stops before physical ANN construction or serving.

Reconstruction pretraining changes what the exported state must carry

Ordinary masked-language-model loss supervises selected token positions; it does not by itself require one sequence row to preserve enough evidence to reconstruct the input. RetroMAE creates that pressure with two differently corrupted views: a full encoder produces a sentence representation, while a shallow decoder reconstructs heavily masked tokens using that representation and its own masked input. The decoder and corruption sampler are training state. The encoder, selected sentence row, and normalization policy are the inference artifact.

RetroMAE v2 / DupMAE changes the exported object again. It trains one reconstruction route from the contextual [CLS] row and a second bag-of-words route from ordinary contextual rows. After dimension reduction and token aggregation, the two routes are concatenated:

$$z_{duplex} = [P_{cls}h_{cls};\operatorname{Agg}(P_{tok}H_{tok})].$$

Discarding the ordinary-token route after training does not produce a smaller equivalent DupMAE artifact; it removes one of the representation channels the objective trained and the paper exports.

C-Pack illustrates a different point: data, benchmark, model family, and training recipe are separate resources. Its BGE lineage composes embedding-oriented pretraining, general contrastive training, and task-specific tuning. A final checkpoint therefore identifies the result of an ordered training program, not every dataset revision, miner, instruction, or intermediate checkpoint that could be paired with the same architecture.

Causal rows begin with unequal information

For a causal backbone,

$$h_t=f_\theta(x_{\le t}),$$

so an early row cannot contain later-token evidence while a final row can see the complete admitted prefix. A plain mean over all causal rows therefore mixes receptive fields. Published methods address this in different places:

Method	Visibility and rendered input	Readout	Training/export consequence
E5-Mistral	causal; task instruction on the query, no instruction prefix on the document; append EOS to both	final-layer EOS row	contrastive LoRA training; preserve side-specific templates, EOS locator, weights/adapter, and normalization
PromptEOL	causal; append an exact semantic-completion prompt	final prompt-token state before generation	can be training-free; the prompt bytes, tokenization, selected row, and layer become representation state
Echo embeddings	causal; render the source twice under a reconstruction/repetition prompt	pool only rows belonging to the second occurrence	preserves the causal model but spends more sequence length; retain the second-copy span map
LLM2Vec	replace causal visibility with bidirectional visibility	mean-pool declared content rows	adapt with masked next-token prediction, then unsupervised contrastive learning; preserve the changed mask behavior and resulting adapter/weights
NV-Embed	remove the causal mask during contrastive training; instruction rows may influence content while remaining outside the final pool	token queries attend to a trainable latent dictionary, pass through an MLP, then mean-pool	retain latent-attention/MLP state, content mask, templates, normalization, and two-stage training lineage
MoEE	use the pretrained MoE model’s visibility and prompt	extract final hidden state plus routing-weight coordinates	concatenation creates one wider vector; weighted score fusion retains two spaces and a fusion coefficient

The E5 contract is a selected-row solution:

$$z_q=H_q^{(L)}[t_{EOS}], \qquad z_d=H_d^{(L)}[t_{EOS}].$$

The EOS row has the whole admitted prefix in its causal receptive field. This does not make the earlier rows bidirectional, and attention_mask.sum(-1)-1 locates EOS only under a compatible right-padded, contiguous-row contract.

Echo instead changes the rendered sequence. For source rows $X$, prefix $P$, and separator $S$:

$$Y=[P,X,S,X], \qquad z_{echo} = \frac{\sum_t m_t^{echo}H_t}{\sum_t m_t^{echo}},$$

where $m^{echo}$ selects content rows in the second occurrence. Pooling both copies restores the early incomplete-view rows that repetition was meant to avoid. Truncating the second occurrence can remove the only rows the readout is allowed to pool.

LLM2Vec changes the attention graph and then adapts it in two distinct stages:

causal checkpoint
    -> bidirectional visibility
        -> masked next-token prediction (token x_i from row h_(i-1))
            -> merge/retain the MNTP adaptation as declared
                -> new SimCSE adaptation over pooled sequence vectors

The previous-row target preserves the decoder’s next-token readout offset. An MLM-style target from the masked row is another objective. Likewise, removing the causal mask without recording the subsequent adaptation does not identify an LLM2Vec release.

NV-Embed learns a pooling block rather than treating a latent array as output queries. With decoder rows $H$ and trainable dictionary $R$:

$$Q=HW_Q, \qquad K=RW_K, \qquad V=RW_V,$$ $$O = \operatorname{softmax}_{r} \left(\frac{QK^T}{\sqrt{d_k}}\right)V.$$

The sequence axis survives in $O$; an MLP and declared masked mean perform the later sequence reduction. Swapping token and latent roles may remain shape-compatible while producing a different object. The paper also changes negative admission between its retrieval-first and mixed-task stages, which is why training-stage identity belongs beside the pooling module.

MoEE exposes another type distinction. Let $e_{HS}$ be a hidden-state vector and $e_{RW}$ the ordered router-weight vector. Concatenation exports one vector,

$$e_{concat}=[e_{HS};e_{RW}],$$

whereas score fusion keeps two geometries:

$$s_{MoEE}(a,b) = s_{HS}(a,b)+\alpha s_{RW}(a,b).$$

The second equation cannot be reproduced by storing one ordinary concatenated vector unless the metric and scaling are transformed to an explicitly proved equivalent form. Layer order, expert-coordinate order, router normalization, selected row, score types, and $\alpha$ are release state. M6.2 owns the router and expert-parallel mechanics; M3.11 owns only the extracted representation and score contract.

Shape-correct substitutions that still change the method

Substitution	Why it is not equivalent
serve RetroMAE’s training decoder as part of every embedding call	training-only reconstruction state becomes an undeclared runtime dependency
export only DupMAE’s [CLS] route	the trained ordinary-token representation channel disappears
apply an E5 query instruction to stored documents	document vectors leave the trained asymmetric space and require re-encoding
select the final padded column instead of the declared EOS/prompt row	width is preserved but row identity changes
pool both Echo copies	rows without full-source causal visibility re-enter the vector
enable bidirectional attention but omit LLM2Vec’s adaptation lineage	an information path is added without identifying weights trained to use it
use in-batch negatives indiscriminately in NV-Embed’s mixed-task stage	same-class examples can be admitted as false negatives
serialize MoEE score fusion as a concatenated-vector index	two separately typed scores and their fusion coefficient disappear

These methods are examples, not a leaderboard. Their durable lesson is that visibility, rendering, row selection, pooling, objective sequence, and exported state move together. Section 0x07 now derives the local training relation; 0x08 reconciles the resulting artifact with batching and export. Physical ANN data structures, shard layout, and search algorithms remain retrieval-system work.

0x07 Training A Readout Means Declaring Gradient Reachability

The forward contracts above do not determine what training is allowed to change. The same classifier can sit on a frozen feature extractor, a trainable pooler over a frozen encoder, a fully fine-tuned encoder, or a parameter-efficient side path. Those programs can produce identically shaped logits while owning different gradients, optimizer slots, checkpoints, and rollback parents.

The original BERT work compared both feature-based use, where contextual states feed a separately trained task model, and fine-tuning, where task parameters and the pretrained parameters are updated jointly. BERT’s reported results are source-reported evidence for those recipes, not proof that full fine-tuning dominates on every dataset. Adapter tuning later inserted small task-specific modules while freezing the shared network, and BitFit studied the narrower choice of updating bias terms. These are different trainable-set contracts, not aliases for “fine-tuning.” (BERT, Houlsby et al., BitFit)

The source-local loss picture is one instance, not the owner contract

Source-local visual from A5 — Training and Reasoning. It usefully separates target and predicted distributions for one vocabulary classification example. M3.11 generalizes the path: the supervised axis may be class, independent label, candidate, token row, or span boundary; its validity mask and reduction denominator must therefore come from the declared task contract rather than from this picture.

Source-to-loss trace: preserve every address change

For one batch, the complete local path is:

source examples + stable example IDs
    -> tokenizer revision + truncation/window policy
        -> input_ids, row_valid, sequence_id, word_id, offsets
            -> backbone(H [B,S,D])
                -> readout R or row-preserving state
                    -> typed head logits Z
                        -> aligned labels Y + supervised mask W
                            -> per-example or per-valid-row losses
                                -> declared reduction numerator S and denominator D_loss
                                    -> scalar L passed to M3.9

The mask used in the loss is not inferred from attention_mask alone. A row can be visible to the encoder and intentionally unsupervised: a special row, the second subword of a word, a question row in extractive QA, or padding in a multiple-choice candidate. Conversely, a sequence-level label supervises one example even though many contextual rows contributed to its pooled readout.

Define a typed per-element loss $\ell_i$, a non-negative supervision weight $w_i$, and a declared denominator:

$$S=\sum_i w_i\ell_i, \qquad D_{loss}=\sum_i w_i, \qquad L=\frac{S}{D_{loss}},$$

with $D_{loss}>0$. For ordinary binary masks, $D_{loss}$ is the number of valid supervised elements. For class weighting or example weighting, it is the sum of weights if that is the chosen estimator. Dividing by padded tensor size, batch size, or sequence length defines a different estimator whenever valid counts vary.

Task addresses make the difference concrete:

task head	loss element address $i$	supervision object	usual local denominator
sequence class/regression	(example)	one class or target vector	valid examples or sum of example weights
multilabel	(example, label)	independent binary targets	valid example-label cells under declared reduction
multiple choice	(example) over candidate axis	correct candidate ID	valid examples; candidate mask acts inside logits
token classification	(example, model_row)	aligned token/word label	supervised model rows, not all visible rows
extractive QA	(example, start/end channel)	two row indices or null rule	valid examples per channel
embedding/retrieval	(anchor, candidate relation)	positive/negative relation	objective-specific admitted anchor or positive count, owned here

M3.11 owns how its head produces typed local losses, which rows reach that head, and which positive/candidate relations enter a contrastive denominator. M3.9 owns microbatch accumulation, distributed global normalization, loss scaling, gradient clipping, optimizer transition, and atomic training checkpoints.

Contrastive readout: the candidate set is part of the objective

A retrieval representation is not trained by pooling alone. Let a readout produce query vectors $q_i$ and document vectors $d_j$, with the normalization and score contract from 0x05. Define:

$$s_{ij}=\operatorname{sim}(q_i,d_j), \qquad \tau>0.$$

For anchor $i$, let $\mathcal A_i$ be the admitted candidate set and $\mathcal P_i\subseteq\mathcal A_i$ its non-empty positive set. A multi-positive contrastive loss is:

$$L_i = -\log \frac{ \sum_{j\in\mathcal P_i}\exp(s_{ij}/\tau) }{ \sum_{k\in\mathcal A_i}\exp(s_{ik}/\tau) }.$$

This equation makes the ownership boundary explicit:

readout contract:
    which contextual rows become q_i and d_j
    whether projection and L2 normalization run
    which similarity creates s_ij

relation contract:
    stable anchor, candidate, and semantic-group IDs
    positive set P_i
    admitted denominator A_i
    hard-negative source and weighting

execution contract:
    batch/device gather order and gradient behavior
    score accumulation dtype and temperature
    local numerator/denominator passed to M3.9

For one positive column $p(i)$, define the admitted softmax:

$$\pi_{ik} = \frac{ \mathbf 1[k\in\mathcal A_i]\exp(s_{ik}/\tau) }{ \sum_{r\in\mathcal A_i}\exp(s_{ir}/\tau) }.$$

Then:

$$\frac{\partial L_i}{\partial s_{ik}} = \frac{1}{\tau} \left( \pi_{ik}-\mathbf 1[k=p(i)] \right).$$

Lower temperature sharpens candidate competition and scales score gradients by $1/\tau$. It is part of training semantics, not display metadata. A document-to-query term may be added:

$$L_{sym} = \frac12 \left( L_{q\rightarrow d}+L_{d\rightarrow q} \right),$$

but the reverse relation is not automatically valid. One document may answer several queries, or one query may have several relevant documents. Treating another true positive as a negative creates a direct force to separate semantically compatible examples.

def contrastive_readout_loss(
    query,
    document,
    positive_mask,
    admitted_mask,
    temperature,
):
    require(query.ndim == document.ndim == 2)
    require(temperature > 0)
    require((positive_mask & ~admitted_mask).sum() == 0)
    require(positive_mask.any(dim=1).all())

    q = l2_normalize(query.float())
    d = l2_normalize(document.float())
    logits = (q @ d.T) / temperature

    denominator = torch.logsumexp(
        logits.masked_fill(~admitted_mask, -torch.inf),
        dim=1,
    )
    numerator = torch.logsumexp(
        logits.masked_fill(~positive_mask, -torch.inf),
        dim=1,
    )
    per_anchor = denominator - numerator
    return per_anchor.sum(), per_anchor.numel()  # numerator/count for M3.9

The positive need not remain on the diagonal after cross-device gathering, sorting, duplicate filtering, or uneven batches. The contract must carry global example IDs and derive positive columns after the final candidate layout. A detached remote candidate still affects the local query through the denominator but may not receive the reciprocal document-encoder gradient. Detached and gradient-preserving gathers are therefore different training programs even when their forward loss values match.

The major published families instantiate this contract differently:

Family	Positive or target relation	Readout/objective consequence
SBERT classification	labeled sentence pair	shared encoder plus declared pool; classifier consumes $[u;v;\lvert u-v\rvert]$
SBERT regression or triplet	similarity target or anchor/positive/negative relation	metric and margin scale are part of the objective
unsupervised SimCSE	two dropout views of the same sentence	view identity and independent stochastic masks define positives
supervised SimCSE	entailment positive with contradiction hard negatives	NLI relation labels alter numerator and denominator membership
DPR-style dual encoder	question with relevant passage and admitted negatives	query/document encoder identity and positive passage IDs define the space

These are not interchangeable recipes that happen to return [B,D]. Changing positive identity, duplicate masking, hard-negative mining, temperature, gather semantics, or normalization produces a new representation training contract.

Alignment and uniformity are diagnostics, not replacement labels

For unit-normalized representation function $f$, a common alignment diagnostic measures the squared distance over the declared positive-pair distribution:

$$L_{align} = \mathbb E_{(x,x^+)\sim p_{pos}} \lVert f(x)-f(x^+)\rVert_2^2.$$

A complementary uniformity diagnostic measures how broadly independent samples occupy the hypersphere:

$$L_{uniform} = \log\mathbb E_{x,y\overset{iid}{\sim}p_{data}} \exp\left(-2\lVert f(x)-f(y)\rVert_2^2\right).$$

These quantities help describe a learned space; they do not decide whether a candidate is relevant. Alignment alone permits collapse, while uniformity alone does not encode the task’s positive relation. Both depend on the sampled positive and population distributions and therefore belong to evaluation evidence for a declared representation release, not to a universal post-processing recipe.

Shape-correct failure	Semantic error	Required evidence
diagonal positive assumed after gather	the target column points to another example	stable global IDs and reconstructed positive map
duplicate positive remains in denominator	a known match is pushed away	group-aware admitted/positive masks
hard-negative weight changes silently	target distribution and gradient scale change	miner and weight revision in the manifest
remote vectors detach unexpectedly	document-side gradient ownership changes	gather backward oracle
all-padding readout is normalized	zero has no valid unit direction	non-empty row admission before pooling/normalization
triplet margin reused under another metric	the margin has another numeric meaning	metric-specific objective fixture

M3.11 owns these semantic relations and local derivatives. M3.9 owns their placement inside accumulation, distributed reduction, optimization, and checkpoint transactions; M4 owns their physical communication and memory cost.

One scalar chain exposes four trainable-state schedules

Use a deliberately small chain so every derivative is visible:

$$h=(\theta+a)x, \qquad r=\phi h, \qquad z=wr, \qquad L=\frac12(z-y)^2.$$

Here $\theta$ is a backbone scalar, $a$ is an optional additive adaptation path, $\phi$ is a learned readout scalar, and $w$ is a task-head scalar. Let $g=z-y$. By the chain rule:

$$\frac{\partial L}{\partial w}=gr, \qquad \frac{\partial L}{\partial \phi}=gwh,$$ $$\frac{\partial L}{\partial \theta}=gw\phi x, \qquad \frac{\partial L}{\partial a}=gw\phi x.$$

The mathematical derivatives exist whether or not a parameter is trainable. Freezing changes which parameter gradients are retained and updated; it does not sever the input-gradient path needed to train a module earlier in the graph. With $x=2$, $\theta=3$, $a=0$, $\phi=0.5$, $w=4$, and $y=10$:

h = 6, r = 3, z = 12, L = 2, g = 2

dL/dw     =  6
dL/dphi   = 48
dL/dtheta =  8
dL/da     =  8

Finite differences in the executable training-state oracle verify all four values. The training schedule then admits only a subset:

schedule	retained trainable gradients	frozen state still used in forward/backward	task artifact
head-only probe	$\nabla_w$	backbone and fixed readout	head plus manifest
learned readout + head	$\nabla_\phi,\nabla_w$	backbone	readout and head
full fine-tuning	$\nabla_\theta,\nabla_\phi,\nabla_w$	none of those three	complete changed model plus head
adapter path + readout + head	$\nabla_a,\nabla_\phi,\nabla_w$	base $\theta$	adapter, readout, head, base identity

Gradient reachability is separate from update ownership. A frozen backbone still executes and may propagate input gradients through its operations. The diagram marks parameter gradients retained by each schedule; M3.9 owns the subsequent optimizer transaction and M6.3 owns the internal adapter map.

Frozen parameters and evaluation mode are independent switches

requires_grad=False prevents accumulation of a parameter gradient. It does not call eval(), disable dropout, freeze batch-dependent statistics in every possible architecture, or make repeated forwards deterministic. These are separate state axes:

def configure_readout_training(model, manifest):
    set_requires_grad(model.backbone, manifest.train_backbone)
    set_requires_grad(model.readout, manifest.train_readout)
    set_requires_grad(model.head, True)

    if manifest.adapter_kind is not None:
        activate_adapter(model, manifest.adapter_release)
        set_requires_grad(model.adapter, manifest.train_adapter)

    model.train(manifest.module_mode == "train")
    assert declared_trainable_names(model) == set(manifest.trainable_names)

A head-only probe can deliberately train with backbone dropout active, but that is a stochastic-feature recipe and must be recorded. If the intended artifact is a deterministic probe over frozen representations, run the backbone in evaluation mode while the new head remains in training mode. A single recursive model.train() may violate that mixed-mode contract.

Backward state follows reachability, not only trainable parameter count

Freezing a backbone removes its parameter gradients and optimizer state, but it does not always remove activation storage. If the first trainable object sits after the entire frozen backbone and no input gradient is needed, the backbone forward can run under a no-grad boundary and expose detached features:

with no_grad():
    H = backbone(batch_inputs)
H = H.detach()
R = learned_readout(H, row_contract)
Z = task_head(R)
L = local_task_loss(Z, labels, supervision_mask)

If a trainable adapter is inserted inside layer 7, backward must reach that layer and the runtime needs the activations or recomputation state required by the adapter path. “Only 1% of parameters are trainable” therefore does not imply “only 1% of training activations are live.” M4 owns the physical memory/FLOP ledger; M3.11 records the first and last trainable graph nodes so that ledger can be derived.

Multiple tasks require normalization before weighting

Suppose task $t$ produces a numerator $S_t$ and denominator $D_t$. First form the task-local estimator, then apply an explicit task weight:

$$L_{multi} = \sum_{t=1}^{T}\lambda_t\frac{S_t}{D_t}, \qquad D_t>0, \qquad \lambda_t\ge0.$$

For a sequence task with two examples and losses summing to $4$, and a token task with four supervised rows and losses summing to $4$:

L_sequence = 4 / 2 = 2
L_token    = 4 / 4 = 1

with lambda_sequence = 1 and lambda_token = 0.5:
L_multi = 1*2 + 0.5*1 = 2.5

Summing all six raw losses and dividing by six would instead optimize an element-weighted mixture. That may be intentional, but it is not equivalent. The task sampler also changes effective weighting: a task selected twice as often contributes twice as many updates unless the schedule compensates.

Alternative adaptation schedules change the hypothesis

method	trainable set	where new computation appears	key tradeoff
frozen feature extraction / linear probe	task head, optionally readout	after backbone	cheap isolation test; representation cannot adapt
full fine-tuning	all declared backbone/readout/head parameters	existing path	broadest local update surface; largest per-task state
gradual or selective unfreezing	changing subset of backbone layers	existing path	schedule and checkpoint phase become part of artifact
BitFit	selected bias terms plus head	existing affine operations	small update set, different hypothesis from low-rank adaptation
bottleneck adapters	inserted task modules plus head	inside blocks	extra nonlinear path; base remains shared
LoRA	selected low-rank weight-update branches plus head	beside chosen linear maps	target modules and rank define update surface

ULMFiT introduced discriminative fine-tuning and gradual unfreezing in a different recurrent-language-model setting; it is useful historical evidence that trainability can be a schedule rather than a fixed set. It is not evidence that the exact ULMFiT recipe transfers unchanged to every Transformer readout. (Howard and Ruder, 2018)

M3.11 compares these schedules by which readout behavior may change. M6.3 derives LoRA factors, initialization, target modules, backward, and adapter checkpoints. This article must not compress those mechanics into a generic “adapter” box.

The training manifest is a proof obligation

training contract:
    dataset revision + split hash + example-ID namespace
    tokenizer/readout/head manifests
    objective kind + label schema + ignored-row policy
    numerator definition + denominator definition
    task weights + task sampler
    trainable-name allowlist + frozen-name hash
    module train/eval modes + dropout/RNG policy
    first and last trainable graph nodes
    base / adapter / readout / head release identities
    parent checkpoint and optimizer-procedure reference -> M3.9

Admission should fail closed:

def validate_training_contract(c, model):
    require(c.objective_kind == model.head.objective_kind)
    require(c.denominator_kind in allowed_denominators(c.objective_kind))
    require(c.valid_supervised_count > 0)
    require(all(weight >= 0 for weight in c.task_weights.values()))

    actual = declared_trainable_names(model)
    require(actual == set(c.trainable_names))
    require(actual.isdisjoint(c.frozen_names))

    if c.schedule == "head_only":
        require(actual <= model.head_parameter_names())
    elif c.schedule == "adapter":
        require(c.adapter_release is not None)
        require(actual <= model.adapter_and_readout_head_names())
    elif c.schedule == "full":
        require(model.backbone_parameter_names() <= actual)
    else:
        require(c.schedule == "readout_head")

    require(c.module_modes_cover_all_stochastic_modules)

The function does not step an optimizer. It proves that the local graph and declared trainable-state policy agree before M3.9 begins the transaction.

Training failure lab

failure	shape check	semantic consequence	detection
attention_mask reused as token-label mask	often passes	special/subword/question rows receive unintended loss	compare row-identity table with supervised-row table
loss divided by padded tensor size	passes	long-padding batches shrink gradient scale	reconcile numerator and declared denominator
frozen backbone left in stochastic train mode accidentally	passes	probe features and metrics vary with dropout RNG	record module modes; replay identical batch
adapter schedule places base weights in optimizer	passes	artifact ceases to be adapter-only	exact trainable-name allowlist
trainable internal adapter executed under no_grad()	passes forward	adapter receives no gradient	first-trainable-node reachability assertion
raw task losses summed before task normalization	passes	tasks with more labels/rows dominate	report each $S_t,D_t,L_t,\lambda_t$
checkpoint saves head without base/readout identity	loads by shape	head is paired with incompatible representation	parent release and manifest hashes
gradual unfreezing resumes at wrong phase	passes	wrong layer subset changes after restart	checkpoint schedule phase and trainable-set hash

The contract closes at a scalar local objective plus typed reachable gradients. The exact update, distributed reduction, finite gate, optimizer state, and restart transaction remain M3.9 responsibilities.

0x08 Runtime Integration Must Preserve The Readout Algebra

A correct training artifact can still be deployed incorrectly. Runtime code adds padding, groups unrelated examples into rectangles, splits long sources into overlapping windows, exports a graph with named dynamic axes, casts tensors, caches representations, and may distribute encoding across workers. Each step can preserve tensor shapes while changing the representation or decision.

The runtime owner question is therefore:

What must remain invariant when one admitted readout artifact is evaluated with different batch shapes, windows, devices, or exported backends?

M3.11 owns the single-artifact answer. M5 owns request scheduling, queueing, continuous batching, and service-level admission. Retrieval systems own ANN index construction, shards, metric realization, and search execution. M4/M8 own physical kernel, precision, compression, and deployment costs.

Padding invariance is a conditional proof

For selected rows $H_s\in\mathbb{R}^{D}$ and binary readout mask $m_s$, masked mean pooling is:

$$r(H,m)=\frac{\sum_{s=1}^{S}m_sH_s}{\sum_{s=1}^{S}m_s}.$$

Append any number of rows with mask zero. If every original valid contextual row remains identical, then:

$$\sum_{s=1}^{S+P}m'_sH'_s=\sum_{s=1}^{S}m_sH_s, \qquad \sum_{s=1}^{S+P}m'_s=\sum_{s=1}^{S}m_s,$$

so $r(H',m')=r(H,m)$. This is the readout proof. It is not yet a complete model proof, because the premise $H'_{f(s)}=H_s$ can fail when:

• padding is visible inside attention;
• left padding shifts absolute position IDs without compensating them;
• the “first row” selector uses physical column zero rather than the first declared valid/special row;
• token-type, sequence, or special-row metadata changes;
• stochastic mode or backend numerics differ.

Official Transformers documentation exposes padding side, padding length, truncation strategy, maximum length, and padding-to-a-multiple as distinct runtime choices. They create rectangular tensors; they do not guarantee semantic equivalence by themselves. (Hugging Face padding and truncation)

For the pooling families in 0x03:

def padding_safe_readout(H, row_valid, kind):
    require((row_valid.sum(axis=1) > 0).all())
    if kind == "mean":
        return (H * row_valid[..., None]).sum(1) / row_valid.sum(1, keepdims=True)
    if kind == "max":
        return where(row_valid[..., None], H, -inf).max(1)
    if kind == "first_valid":
        index = first_true_index(row_valid, axis=1)
        return gather_rows(H, index)
    raise ValueError("unsupported readout")

Zero-filling invalid rows before max remains wrong when all valid values in a channel are negative. Selecting H[:, 0] remains wrong under left padding. The runtime oracle evaluates the same valid rows with left and right padding and proves that all three corrected readouts agree.

Dynamic batching changes allocation, not example identity

One request can move among batches of different width. Its output is admitted only if the runtime preserves:

example_id
source rendering and normalization
tokenizer revision
content token IDs and row identities
valid-row / readout masks
position and segment policy
readout/head/calibrator/decision releases
stochastic and dtype policy

Batch order is not example identity. Every output row must carry the stable example ID through sorting, bucketing, padding, model execution, and scatter back. A length-sorted batch that returns outputs in sorted order without the inverse permutation is numerically plausible and semantically wrong.

Long sources require an explicit window contract

Truncation is one policy, not a neutral preprocessing step. Official tokenizer APIs expose overflowing encodings, offsets, sequence IDs, special-token masks, and a stride that repeats content from the preceding piece. Those fields are enough to build an auditable window map; they do not choose how predictions from those windows should be reconciled. (Tokenizers Encoding, Transformers tokenizer API)

An immutable window record needs:

source_id, window_id, parent release
source character interval and source token identities
window-local row -> source token / character offset
left/right context and stride
special-row insertion
overlap multiplicity or one-owner assignment
task-specific aggregation policy

There are several non-equivalent design choices:

long-source policy	retained evidence	main failure boundary
hard truncation	one prefix/suffix/pair allocation	omitted evidence is unrecoverable
overlapping windows	local context around all covered spans	duplicate source rows and inconsistent boundary context
hierarchical readout	one vector/logit per chunk, then document aggregator	aggregator is another trained or declared model
late interaction	many selected row vectors	larger persistent artifact; retrieval systems own index realization
long-context backbone	one longer contextual graph	different model/position/runtime contract, not “no chunking” for free

Overlap correction fixes ownership, not context

Let occurrence $(j,s)$ in window $j$ map to source token $u(j,s)=i$. Let $c_i$ be the number of retained occurrences of source token $i$. A de-duplicated mean over occurrence representations is:

$$r_{overlap} = \frac{1}{N_{unique}} \sum_{j,s} \frac{m_{j,s}}{c_{u(j,s)}}H_{j,s}.$$

For source values [2, 4, 10, 20] and windows [2,4,10] and [10,20], the naive mean of window means is:

((2 + 4 + 10) / 3 + (10 + 20) / 2) / 2 = 10.1667

The multiplicity-corrected occurrence mean is:

(2 + 4 + 10/2 + 10/2 + 20) / 4 = 9

which matches the unique-value mean. But contextual rows for the repeated token need not be equal across windows: each occurrence sees different neighbors and positions. Multiplicity correction averages those occurrence representations; it cannot reconstruct the representation from one unavailable full-context forward. Choosing a “maximum-context” owner, center-weighting, first owner, or learned hierarchy defines another contract.

Aggregate the object the model was trained to aggregate

For document classification, common policies include averaging logits, averaging probabilities, taking a maximum, or applying a trained second-stage aggregator. They are not interchangeable. With chunk logits $z_j$:

$$p_{logit}=\operatorname{softmax}\left(\frac1J\sum_j z_j\right), \qquad p_{prob}=\frac1J\sum_j\operatorname{softmax}(z_j).$$

Softmax is nonlinear, so generally $p_{logit}\ne p_{prob}$. For chunk logits [4,0] and [0,0]:

softmax(mean logits)     ~= [0.8808, 0.1192]
mean(softmax(each logit))~= [0.7410, 0.2590]

Both predict class zero here, but their confidence and threshold behavior differ. Calibration must be fitted after the aggregation point used in deployment. A per-chunk temperature cannot be assumed valid for a document-level maximum or mean.

Task-specific reconciliation differs again:

output	window reconciliation
token labels	map rows through offsets; resolve duplicate token ownership before metrics/output
QA spans	map local start/end to source offsets; reject cross-window spans; compare null and span scores under one declared rule
sequence logits	aggregate logits/probabilities with a trained or fixed policy before final decision
embedding vector	aggregate chunk vectors or token occurrences, then normalize at the trained stage
late-interaction rows	de-duplicate or retain versioned occurrences under the artifact schema

For normalized embeddings, “normalize each chunk then average” differs from “average unnormalized chunk vectors then normalize.” The runtime must match the training and evaluation geometry, not merely return unit length.

Export is an ABI, not a file conversion

An exported readout graph must name both its axes and semantics:

inputs:
    input_ids       int64 [batch, sequence]
    attention_mask  bool/int [batch, sequence]
    token_type_ids  optional, model-specific [batch, sequence]
    position_ids    optional, policy-specific [batch, sequence]

outputs, exactly one declared family:
    sequence_logits [batch, classes]
    regression      [batch, outputs]
    token_logits    [batch, sequence, classes]
    start_logits + end_logits [batch, sequence]
    embeddings      [batch, embedding_dim]
    token_vectors   [batch, sequence, projection_dim] + row mask

Batch and sequence may be dynamic axes. Class, output, embedding, and projection dimensions are release constants. Dynamic shape support can change backend optimization opportunities, so “dynamic” is both an ABI and performance choice. ONNX Runtime documentation explicitly notes graph input/output names, shapes, and dynamic axes as export-visible properties and warns that dynamic axes can constrain some Transformer optimizations. (ONNX Runtime PyTorch inference, Transformer optimization)

Do not export a bare output tensor and reconstruct semantics from its rank. [B,S,C] could be token classes, token vectors, or another hidden state. The release must carry output kind, score type, row mask, normalization stage, calibrator, threshold/tie policy, and label map.

Reusable state is keyed by the complete representation release

A cached embedding or token-vector artifact is valid only under a complete key:

$$K_{state} = H(\text{source identity},\text{rendering},\text{tokenizer},\text{backbone}, \text{adapter},\text{row policy},\text{readout/projection},\text{dtype schema}).$$

Changing a calibrator may leave embeddings reusable but invalidate final decisions. Changing the readout or normalization invalidates vectors. Changing a K/V-targeting adapter can invalidate contextual rows. Cache identity should therefore be layered rather than one opaque “model version” string:

contextual-state release -> readout/vector release -> score release -> decision release

Cross-encoders have no query-independent document contextual state. Bi-encoders can cache one document vector. Late-interaction systems can cache selected document row vectors. The cacheability statement follows the interaction graph from 0x06, not a runtime optimization toggle.

Distributed encoding needs algebraic merge rules

Independent examples can be encoded on different workers and scattered back by stable ID. Splitting one example across workers is harder: only readouts with an associative sufficient state can merge exactly after contextual rows already exist. For masked mean, worker $r$ can return:

$$S_r=\sum_{s\in\mathcal S_r}m_sH_s, \qquad n_r=\sum_{s\in\mathcal S_r}m_s,$$

then:

$$r=\frac{\sum_r S_r}{\sum_r n_r}.$$

For max, merge channel-wise maxima and retain deterministic argmax ownership if backward or provenance is needed. Learned attention cannot generally normalize each shard independently and average the outputs; it must merge the global softmax sufficient state or evaluate one global denominator. Attention inside the backbone also cannot be recreated by independently encoding arbitrary sequence shards unless the backbone architecture itself defines such a split.

M4 owns communication bytes and collective placement. M5 owns worker admission and scheduling. M3.11 only declares the mathematical merge object and stable output identity.

The runtime may change rectangular allocation and physical placement. It may not silently change row identity, chunk ownership, output type, or the release key of reusable representations.

Runtime admission and failure lab

def admit_readout_request(request, release):
    require(request.release_id == release.id)
    require(request.tokenizer_revision == release.tokenizer_revision)
    require(request.output_kind == release.output_kind)
    require(request.padding_side in release.tested_padding_sides)
    require(request.max_length <= release.exported_max_length)

    if request.requires_overflow:
        require(release.window_policy is not None)
        require(release.window_aggregation is not None)
        require(request.offsets_available)

    if release.output_kind in {"embedding", "token_vectors"}:
        require(request.normalization_stage == release.normalization_stage)
    return True

failure	shape check	consequence	detection
left padding with unshifted absolute positions	passes	valid contextual rows change	compare unpadded/left/right outputs by stable row ID
H[:,0] used as first-row readout	passes	left pad becomes representation	first-valid/special-row identity assertion
overlapping windows counted twice	passes	boundary evidence gets extra weight	source-token multiplicity ledger
chunk probabilities averaged after logit calibration	passes	confidence contract changes	aggregate-point identifier and calibration replay
QA span start/end taken from different windows	passes indices	impossible source span	require shared source/window compatibility
export drops row mask for token vectors	passes tensor rank	padded rows enter late interaction	output ABI requires companion row mask
cached vector reused after readout revision	passes dimension	stale representation in new score space	layered release key
worker averages local means equally	passes	unequal shard lengths bias global mean	merge (sum,count), not local means
sorted batch is not inverse-permuted	passes	outputs assigned to wrong examples	stable-ID scatter reconciliation

The next section pins official implementations and reconciles these mathematical contracts with concrete source symbols. The runtime derivation above deliberately stops before framework-specific class names become claims.

0x09 Pinned Implementations Expose Hidden Policy

Paper equations describe a family of implementations. A moving documentation page or main branch cannot prove what one release did. This section therefore uses immutable source snapshots captured on 2026-08-05:

project	pinned revision	local evidence
Hugging Face Transformers	0104732151118872214a2bcdc5badd32d938ab38	hf_modeling_bert_0104732.py
Sentence Transformers	f6d3a4030d6878b3d916971de4686cef46626683	pooling, normalization, and cross-encoder snapshots

The files and SHA-256 digests live under docs/numinens-evidence/m3-11-representation-readout/sources/. They are evidence snapshots, not vendored dependencies. Statements below describe these commits. Earlier or later releases may choose another class layout, default, or edge-case policy.

BertPooler is a learned first-physical-row map

The pinned BertPooler performs:

first_token_tensor = hidden_states[:, 0]
pooled_output = self.dense(first_token_tensor)
pooled_output = self.activation(pooled_output)  # tanh

Its durable tensor contract is:

$$p=\tanh(W_pH_{:,0,:}+b_p), \qquad p\in\mathbb{R}^{B\times D}.$$

Three policies are bundled:

1. the selected row is physical column zero;
2. the map has learned $D\times D$ weight and bias;
3. the output is bounded by tanh.

That is not equivalent to raw [CLS], first-valid pooling, or masked mean. It is safe under the ordinary right-padded BERT convention because the special classification token occupies row zero. If a runtime left-pads without keeping that token at row zero, the pooler does not search the mask for it.

The sequence-classification wrapper owns more than a linear layer

At the pinned revision, BertForSequenceClassification instantiates a BertModel with the pooler, applies classifier dropout, and maps the pooled width to num_labels:

input rows
    -> BertModel
        -> pooler_output [B,D]
            -> classifier dropout
                -> Linear(D, num_labels)
                    -> logits [B,num_labels]

When config.problem_type is absent, the wrapper infers a local objective from label count and dtype:

num_labels == 1                   -> regression / MSE
num_labels > 1 and integer labels -> single-label / cross-entropy
otherwise                         -> multilabel / BCE-with-logits

This convenience is version-specific and can make dtype part of objective selection. A float tensor containing class IDs can therefore choose a different loss path than an integer tensor of the same shape. M3.11’s release manifest is stricter: problem_type, label schema, and output meaning are declared before the batch arrives rather than inferred from one batch dtype.

Token classification preserves rows but does not align source words

BertForTokenClassification constructs BertModel(..., add_pooling_layer=False), then applies dropout and one classifier to every model row:

$$Z_{b,s,:}=W_c\operatorname{Dropout}(H_{b,s,:})+b_c.$$

Its loss flattens logits to [-1, num_labels] and labels to [-1] before cross-entropy. The wrapper does not derive word_ids, choose first-subword ownership, or map results to source offsets. Those remain preprocessing and release contracts. The common -100 ignore behavior comes from the loss implementation’s default; it does not make every attention-masked row ignored automatically.

This snapshot therefore agrees with the 0x04 tensor trace while reinforcing a boundary: a task-head class consumes aligned labels; it does not prove their alignment.

The QA wrapper creates two channels, then chooses an out-of-range sentinel

The pinned BertForQuestionAnswering disables the BERT pooler and applies one linear map to every contextual row. It splits the final channel into start and end logits:

H [B,S,D] -> Linear(D,2) -> [B,S,2]
                         -> start [B,S], end [B,S]

For training, it defines ignored_index = S, clamps supplied positions into [0,S], applies cross-entropy to each channel with that ignored index, then averages start and end losses. Row index $S$ is outside the score axis 0..S-1, so it acts as an ignore sentinel rather than a candidate span row.

The implementation does not enforce start <= end, context-segment ownership, maximum answer length, or a no-answer score rule. Those are decode and dataset contracts from 0x02/0x04. Reading only the class name would hide this separation.

Sentence Transformers pooling is a configurable concatenation

The pinned Pooling module accepts one or more modes:

cls, max, mean, mean_sqrt_len_tokens, weightedmean, lasttoken

Every selected mode emits a vector, and the module concatenates them on the last axis. Therefore two active width-$D$ modes emit width $2D$; the output is not an average of modes.

The implementation reconciles with the derivations in 0x03 as follows:

mode	pinned implementation	important edge policy
cls	uses supplied cls_token_embeddings, else gathers first 1 in attention mask	fallback supports left padding; it identifies first valid, not necessarily a semantic CLS row
max	fills masked entries with -inf, then channel-wise max	agrees with the negative-value counterexample
mean	masked sum divided by mask sum	denominator is clamped to 1e-9 rather than rejecting an empty row set
mean_sqrt_len_tokens	same sum divided by square root of clamped count	different length scaling, not ordinary mean
weightedmean	physical weights 1..S times attention mask	left padding shifts content weights unless the intended policy accounts for it
lasttoken	finds last true mask entry by reversing the mask	all-zero fallback returns a masked zero row instead of fail-closed rejection

The article’s zero-valid-row admission is deliberately stricter than these fallbacks. Returning a finite zero or clamped result can keep a batch running, but it also hides malformed row ownership. Whether to reject or tolerate that case belongs in the release contract.

The physical position weighting deserves emphasis. With right padding, content rows at columns 0,1,2 receive weights 1,2,3. Move the same content to columns 2,3,4 under left padding and the weights become 3,4,5. Masking removes pads but does not restore content-relative positions. Therefore the pinned weightedmean mode is not left/right padding invariant in general.

Normalization is a separate module and therefore a separate stage

The pinned Normalize module reads sentence_embedding and applies:

F.normalize(sentence_embedding, p=2, dim=1)

It does not normalize token embeddings before pooling, infer a score metric, or attach a threshold. Module order determines geometry:

Transformer -> Pooling -> Normalize

is generally different from:

Transformer -> normalize each token -> Pooling

and both differ from document-window aggregation followed by normalization. The serialized module graph, not the presence of a Normalize class anywhere in the repository, defines the release.

CrossEncoder prediction exposes score-policy and permutation contracts

At the pinned Sentence Transformers revision, CrossEncoder.predict:

1. moves the model to the requested device and calls eval();
2. sorts input pairs by estimated length;
3. preprocesses each joint pair and runs the model;
4. applies a configured activation;
5. optionally applies softmax for multi-output scores;
6. squeezes the single-label axis when applicable;
7. inverse-permutes results to original input order.

The default activation is sigmoid when num_labels == 1 and identity otherwise; apply_softmax defaults to false. Consequently, the API can return a sigmoid-transformed scalar for one model and raw multi-class logits for another. “CrossEncoder score” is not one universal score type.

The inverse permutation mechanically supports 0x08’s batch-identity rule. It uses original list indices as the local identity; a larger service still needs stable request/example IDs around that call. Joint pair preprocessing also confirms the architecture boundary: document state depends on the query and is not independently reusable.

Pinned source symbols are evidence for one revision. The article extracts durable tensor and semantic contracts, then records where convenience defaults are weaker than release admission.

Executable source reconciliation

The pinned-source oracle does not import these frameworks. It instead:

verifies each pinned file SHA-256
checks exact semantic source markers
rejects HTML/404 bodies and unpinned revisions
reuses prior numeric pooling/head/runtime oracles
emits one source-symbol -> article-contract report

The checked markers include:

BertPooler: hidden_states[:, 0] -> dense -> tanh
sequence wrapper: pooled output -> dropout -> classifier
token wrapper: sequence output -> dropout -> classifier
QA wrapper: split start/end; ignore sentinel at sequence length
Pooling: -inf max; sum/count mean; first-valid fallback
Normalize: p=2 along sentence embedding dimension
CrossEncoder: eval; length sort; activation; optional softmax; inverse permutation

This is stronger than citing a documentation page and weaker than reproducing all upstream tests. It proves the narrow implementation claims made here while leaving kernel/runtime behavior to its owning modules.

0x0A Evaluation And Rollback Must Preserve The Semantic Tuple

An output tensor is not publishable because it has the expected shape, and a readout is not releasable because one aggregate score improved. Evaluation must exercise the exact tokenizer, contextual state, readout, head, score, calibrator, decision, runtime, and window policy that will be released. A rollback must restore that complete semantic tuple or explicitly migrate every dependent artifact.

This section owns readout-local semantic evidence. It does not duplicate M8.10, which owns fleet workloads, latency/goodput, resource and energy gates, shadow traffic, canary attribution, production monitoring, and operational rollback. M3.11 produces the semantic candidate and compatibility record consumed by that deployment process.

Freeze split roles before observing final outcomes

One immutable split manifest needs more than filenames:

dataset revision and license/provenance
stable source/group IDs and deduplication policy
train, development, calibration, promotion-test roles
window/chunk parent identities
label schema and scorer revision
slice definitions fixed before final evaluation
allowed tuning operations per split

The roles are distinct:

split	permitted use	forbidden use
train	fit backbone/readout/head/adapters	report as held-out evidence
development	choose architectures and ordinary hyperparameters	final promotion claim after repeated tuning
calibration	fit temperature, thresholds, abstention, or null-score rules	update backbone/head from those outcomes unless reclassified as train data
promotion test	evaluate one frozen candidate and policy	tune the candidate, thresholds, slices, or margins after seeing results

All windows from one source document, conversation, user, patient, or other declared dependence group must stay in one role. Splitting overlapping windows at random creates nearly duplicated train/test evidence. M4.7-M4.9 own the general scaling-law split discipline; M3.11 applies the same immutable-identity principle to readout evaluation.

Match the metric to the output object

No one metric covers all heads:

output contract	primary semantic metrics	required companion evidence
single-label class	accuracy; macro/micro precision, recall, F1	confusion matrix, per-class support, calibration and cost-sensitive errors
multilabel	per-label and macro/micro F1; average precision where appropriate	prevalence, threshold vector, labels per example, empty-label policy
regression	MAE/MSE/RMSE; Pearson or Spearman when the task defines it	target range, residual slices, clipping and missing-target rules
token/entity labels	exact entity-span precision/recall/F1	token accuracy, illegal tag sequences, boundary/type error table, offset scorer revision
extractive QA	normalized exact match and token-overlap F1	answerable/unanswerable slices, null-score rule, window coverage and truncation failures
sequence similarity	task-defined correlation or pair classification	normalization/metric contract and score distribution
retrieval/reranking	recall, MRR, nDCG, MAP under fixed candidates/corpus	representation and candidate semantics are local; index/search execution is owned by retrieval systems

GLUE’s value is its multi-task and diagnostic framing, not one universal aggregate. MTEB similarly exists because performance on one embedding task such as semantic textual similarity does not establish performance on clustering, classification, reranking, or retrieval; its original paper reports no single method dominating every evaluated task. These are externally verified reasons to preserve task buckets, not claims that either benchmark is a sufficient production gate. (GLUE, MTEB)

For exact entity evaluation, a predicted span is a true positive only when its source boundary and type match under the pinned scorer. If $TP$, $FP$, and $FN$ count exact entities:

$$P=\frac{TP}{TP+FP}, \qquad R=\frac{TP}{TP+FN}, \qquad F_1=\frac{2PR}{P+R}.$$

Token accuracy can remain high while entity F1 collapses on boundaries. CoNLL evaluation history also illustrates that scorer behavior for malformed tag sequences matters; record the exact scorer rather than reimplementing an informal “CoNLL F1.” (CoNLL-2003 shared task, NER scorer reproducibility analysis)

Macro and micro answer different weighting questions

For class $c$, define $TP_c,FP_c,FN_c$ and $F_{1,c}$. Macro F1 gives every class equal weight:

$$F_{1,macro}=\frac1C\sum_{c=1}^{C}F_{1,c}.$$

Micro F1 first sums event counts:

$$F_{1,micro} = \frac{2\sum_cTP_c} {2\sum_cTP_c+\sum_cFP_c+\sum_cFN_c}.$$

For single-label multiclass classification, micro F1 equals accuracy; macro F1 can expose failure on rare classes. In the evaluation fixture, six examples yield:

class 0: F1 = 1.0
class 1: F1 = 0.5
class 2: F1 = 0.5

macro F1 = 2/3
micro F1 = accuracy = 4/6

The equality here is a property of the single-label counting scheme, not a universal identity for multilabel or token tasks.

Calibration and decisions need separate held-out evidence

Accuracy and ranking can remain fixed while confidence changes. A classification release should preserve at least:

raw logits
post-calibration probabilities
predicted class or thresholded labels
negative log-likelihood / Brier score
calibration summary with bin definition
coverage and risk under abstention
cost-weighted false positive/negative counts

Temperature, thresholds, per-label thresholds, and abstention regions are fit on the calibration split and frozen before promotion testing. If the threshold is changed after inspecting test errors, the old test set has become tuning data; a new untouched gate is required.

For a confidence partition into bins $B_k$, expected calibration error is often reported as:

$$\operatorname{ECE} = \sum_k\frac{|B_k|}{N} \left|\operatorname{acc}(B_k)-\operatorname{conf}(B_k)\right|.$$

ECE depends on bin construction and can hide within-bin or slice errors. It is a diagnostic with a versioned estimator, not a universal proof of calibration. NLL, Brier score, reliability plots, and critical-slice calibration should accompany it when confidence drives actions.

Slices make failure mechanisms visible

Freeze slices from mechanism hypotheses before the final gate:

source length and truncation/window coverage
left/right padding and batch-width buckets
language, domain, time, and source system
class/label support and multi-label cardinality
entity type and span length
QA answerability, answer length, and answer position
special-token, subword, Unicode, and offset complexity
cross-encoder pair length and candidate rank
embedding norm, query/document length, and interaction family

Report support and uncertainty with every slice. A perfect score on three cases is not evidence of a robust subgroup. Global metrics must not average away a critical slice floor, and slices must not be invented post hoc until one makes the candidate look favorable.

Pair candidate and baseline on stable case identities

Evaluate the candidate and rollback parent on the same stable cases and retain per-case outputs:

$$\Delta_i=m_i(candidate)-m_i(parent).$$

Pairing removes variation from changing examples and supports direct error transition tables:

both correct
parent correct -> candidate wrong
parent wrong -> candidate correct
both wrong, same or different failure class

M8.10 owns formal promotion margins, paired uncertainty, non-inferiority, and deployment selection. M3.11 supplies stable semantic case IDs, outputs, and failure categories so those tests operate on the right objects.

Compatibility is a dependency graph, not one Boolean

Represent the artifact as layers:

source/tokenizer
    -> contextual state
        -> readout/vector
            -> head/raw score
                -> calibration
                    -> decision/label

A change invalidates its descendants unless an explicit migration proves compatibility:

change	contextual cache	vector cache	raw scores	calibrated scores	decisions
tokenizer/template/position policy	invalidate	invalidate	invalidate	invalidate	invalidate
backbone or active internal adapter	invalidate	invalidate	invalidate	invalidate	invalidate
selected layer/pooling/projection/normalization	may reuse contextual rows if identity matches	invalidate	invalidate	invalidate	invalidate
task head weights or ordered label map	reusable	reusable if head consumes vectors	invalidate semantic score artifact	invalidate	invalidate
calibrator only	reusable	reusable	reusable	invalidate	invalidate
threshold/tie/abstention only	reusable	reusable	reusable	reusable	invalidate
backend/dtype/export	compatibility unproven until parity tests	same	same	same	same

“Reusable” means semantically eligible under the exact parent identities; it does not mean retaining the cache is operationally worthwhile. Backend/dtype changes are not automatically invalid, but they require parity evidence under the numerical contract before sharing a namespace.

Migration must be explicit and reversible

Examples:

• adding a new label changes output width and normally requires a new head;
• reordering labels with unchanged width changes coordinate meaning and must migrate both head rows and label maps together, or reject the artifact;
• changing mean pooling to concatenated mean+max changes head input width;
• changing calibration preserves raw logits but creates a new calibrated-score and decision release;
• changing window stride changes which evidence reaches a document-level readout even when tensor shapes stay fixed.

A migration record needs source and target release IDs, transformed objects, preconditions, postconditions, parity tests, cache invalidation plan, and a rollback target. “Load succeeded” proves only structural compatibility.

Local lifecycle and deployment lifecycle meet at the candidate boundary

M3.11’s semantic artifact state machine is:

DRAFT
    -> RECONCILED       hashes, schemas, source markers, local oracles pass
        -> EVALUATED    frozen semantic suite and critical slices pass
            -> CANDIDATE complete package, compatibility and rollback records

M8.10 consumes CANDIDATE and owns shadow, canary, ramp, active production, monitoring, and fleet rollback. If a deployed readout fails, the M3.11 rollback record tells M8.10 which semantic tuple and cache namespaces constitute the known-good parent.

M3.11 proves the semantic candidate. M8.10 decides whether and how that candidate receives production traffic. Rollback joins both contracts: restore the known-good semantic tuple and execute the service transition safely.

Rollback restores a known-good tuple, not just old weights

The rollback parent must include:

tokenizer/template and source normalization
backbone, adapter, selected layer, and dtype policy
readout/pooling/projection/normalization
head, ordered label map, null label, score type
calibrator, thresholds, abstention, and tie rules
window/export/runtime schemas
compatible cache namespaces or explicit invalidation
evaluation evidence and known-good release ID

Restoring only the old checkpoint while leaving the new tokenizer, label order, pooling configuration, calibrator, or vector cache can reproduce the original failure under an old model filename. Rollback is a forward transition to a complete known-good identity.

Publication evidence bundle

manifest/
  semantic artifact and parent identities
  immutable split and scorer manifests
  trainable/runtime/export contracts

evaluation/
  per-case outputs for candidate and parent
  aggregate, per-class, per-label, span, QA, calibration metrics
  frozen slices, supports, failure taxonomy

implementation/
  pinned source revisions, snapshots, hashes, marker report
  numerical oracle and backend parity reports

lifecycle/
  migration and compatibility matrix
  invalidation plan and rollback target
  M8.10 deployment handoff

Final readout failure lab

layer	shape-correct failure	evidence that catches it
input	tokenizer or template drift	stable source-to-row trace and artifact hash
rows	special/padding/subword row enters wrong readout/loss	row identity and offset oracle
pooling	zero-filled max or wrong denominator	negative-value and padding-invariance fixtures
head	label order changes at same width	ordered map plus semantic probes
span	local window indices treated as source offsets	overflow/offset reconciliation and exact spans
score	sigmoid probability treated as raw logit	score-type and pinned CrossEncoder marker
contrastive objective	positive mapping or admitted denominator changes silently	stable relation IDs, candidate masks, and gather-gradient oracle
calibration	threshold tuned on test	immutable split-role audit
batching	sorted outputs assigned to wrong examples	stable-ID inverse-permutation check
chunking	overlap counted twice or logits/probabilities swapped	multiplicity and nonlinear aggregation oracle
cache	old vector reused after readout change	compatibility dependency graph
migration	head loads against incompatible pool width	derived width and migration preconditions
rollback	old weights restored with new label map/cache	complete known-good tuple reconciliation

The article is publishable only when these failures are represented by executable checks, source evidence, or explicit release gates rather than by warnings alone.

0xFF References

• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• SimCSE: Simple Contrastive Learning of Sentence Embeddings
• Dense Passage Retrieval for Open-Domain Question Answering
• ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
• RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encoder
• RetroMAE v2: Duplex Masked Auto-Encoder For Pre-Training Retrieval-Oriented Language Models
• C-Pack: Packed Resources For General Chinese Embeddings
• Improving Text Embeddings with Large Language Models
• Scaling Sentence Embeddings with Large Language Models
• Repetition Improves Language Model Embeddings
• LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
• NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models
• Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Free
• Massive Text Embedding Benchmark
• Hugging Face Transformers BERT documentation
• Sentence Transformers pooling modules
• Hugging Face Tokenizers Encoding API
• Hugging Face token-classification guide
• Hugging Face extractive question-answering guide
• A Structured Self-Attentive Sentence Embedding
• On Calibration of Modern Neural Networks
• Understanding Contrastive Representation Learning through Alignment and Uniformity
• Sentence Transformers Cross-Encoder documentation
• Parameter-Efficient Transfer Learning for NLP
• BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
• Universal Language Model Fine-tuning for Text Classification
• Hugging Face padding and truncation
• Hugging Face tokenizer API
• ONNX Runtime PyTorch inference
• ONNX Runtime Transformer optimization
• GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
• Introduction to the CoNLL-2003 Shared Task
• If You Build Your Own NER Scorer, Non-replicable Results Will Come
• Transformer Series (2) — Overall Architecture
• Transformer Series (4) — Encoder and Decoder
• Transformer Series (5) — Training and Reasoning
• Transformer Series (6) — Tokenization
• M8.10 — Deployment Evaluation, Release, and Rollback