M4.9 - Over-Training, Downstream Metrics, and Lifecycle Allocation

0x00 From Training-Optimal To Lifecycle-Optimal

M4.7 derives an allocation under a training-compute constraint. M4.8 asks whether the fitted surface survives hidden scales, alternative curve families, repeated data, and dependence. Neither article yet decides which artifact should be trained and operated for a real workload.

That last step changes the objective. A model can be suboptimal for minimizing training loss at one fixed training budget and still be preferable over its operational lifetime. A smaller model trained on more tokens may cost more to train than the cheapest route to one validation loss, yet cost less to serve millions of times. A candidate with slightly better aggregate benchmark score may fail one critical task tail. A quantized artifact may preserve validation loss while missing latency, memory, calibration, or rollback requirements.

This creates M4.9’s owner question:

When should a model be trained beyond its training-compute optimum, and how should validation loss, downstream quality, serving demand, runtime cost, uncertainty, and release evidence determine the final artifact?

The thesis is that deployment allocation is a discrete, uncertainty-aware lifecycle decision, not another smooth scaling-law optimum. The decision must preserve the provenance of each input: M4.8 supplies certified laws, the frozen estimator that refits them, and its own prediction draws (Section 0x02M explains which draws set the quality risk), M5 supplies measured runtime behavior, and M8 supplies qualified artifact evidence. M4.9 combines those inputs without replacing their mechanism-level owners.

0x00A Three Frontiers Answer Different Questions

The word optimal is incomplete unless its constraint and accounting boundary are named.

Frontier	Decision question	Principal coordinates	Required evidence	Characteristic failure
training-compute frontier	for a fixed training budget, which model/token allocation minimizes the registered validation loss?	$N$, $D$, training compute, loss	M4.7 measured and fitted run family	treating one paper’s constants as universal
fixed-quality size/token frontier	for a target quality, how much extra training can buy a smaller model?	model size, token multiplier, quality tolerance	M4.8-supported performance surface and downstream mapping	calling every large D/N value beneficial over-training
lifecycle artifact frontier	which discrete trained and packaged artifact minimizes expected lifetime cost while satisfying quality, capacity, reliability, and release gates?	training, prefill, decode, demand scenarios, memory, artifact variant	M4.8 prediction draws, M5 measurements, M8 qualification	optimizing an analytic proxy that the deployed system does not realize

These frontiers can select different candidates without contradiction. The training-compute frontier asks how to spend a run budget. The fixed-quality frontier asks whether extra tokens can trade training work for fewer served parameters. The lifecycle frontier asks whether that trade remains favorable after the actual prompt lengths, generated lengths, batching, K/V pressure, hardware efficiency, quantized artifact, failure policy, and demand horizon are included.

0x00B Loss Is Not Product Utility

Validation cross-entropy is a useful common coordinate because it is smooth, dense, and available across training runs. It is not a complete product objective. The map from loss to downstream behavior may depend on task, prompting, decoding policy, metric definition, and score aggregation.

The evidence chain must remain visible:

training configuration
    -> validation loss distribution
    -> per-task model outputs
    -> registered per-task metrics
    -> aggregate and tail-quality constraints
    -> release decision

A thresholded metric can display an apparent jump even when the underlying probabilities move smoothly. An aggregate can improve while a safety-critical or revenue-critical task regresses. A fitted loss-to-score relation can be useful inside measured support without proving that every capability follows the same curve.

M4.9 therefore keeps three objects rather than one benchmark number:

Evaluation object	Why it remains separate
registered aggregate	supports a stable headline comparison across candidates
per-task and subgroup distribution	exposes regressions hidden by averaging
metric-sensitivity audit	tests whether a threshold, discontinuity, or aggregation rule created the apparent behavior

The later derivation will show how over-training changes the performance surface. It will not promote validation loss into a universal utility score.

0x00C Serving Cost Is A Measured Artifact Property

Lifecycle accounting needs a resource model, but no one source supplies the whole serving cost:

M4.1 analytic ledger:
    expected parameters, FLOPs, activations, transfers, and K/V bytes

M5 runtime evidence:
    admission, prefill/decode scheduling, batching, cache pressure,
    placement, throughput, latency distributions, and failure behavior

M8 artifact evidence:
    quantized representation, kernels, accuracy deltas, memory footprint,
    compatibility, release manifest, and rollback artifact

M4.9 consumes these measurements as versioned evidence. It does not infer production latency from FLOPs alone, price a request from parameter count alone, or assume that a quantized checkpoint realizes the same kernels on all hardware.

Training, prefill, and decode also remain separate ledgers. They have different amortization and bottlenecks:

Cost phase	Primary driver	Why aggregation too early is dangerous
training	run family, token count, optimizer, parallelism, failures	a one-time cost can dominate a short-lived model but vanish over a large demand horizon
prefill	prompt shape, batching, attention/backend path	prompt-heavy traffic can favor a different artifact than decode-heavy traffic
decode	generated tokens, active sequences, K/V state, memory bandwidth	average FLOPs can hide tail latency, fragmentation, or low utilization
release and rollback	qualification runs, canary capacity, retained fallback	the cheapest steady-state artifact may be expensive or unsafe to reverse

Only after these ledgers are measured under compatible scenarios may they be combined into lifecycle cost.

0x00D The Decision Object Is Discrete And Reversible

A production choice is not an unconstrained real-valued pair $(N,D)$. It is a finite artifact and runtime configuration:

checkpoint and training horizon
precision / quantization representation
parallelism and backend
hardware pool and replica count
batching, admission, and K/V policy
quality thresholds and exception policy
traffic scenario and service-level objective
assumed release sequence and rollback target (requirements handed downstream)

The optimization should enumerate feasible candidates, evaluate each one inside uncertainty draws and demand scenarios, reject constraint violations, and compare the survivors. Integer replica counts, memory admission, and capacity rounding occur inside each draw. Rounding only the final mean can turn an infeasible candidate into an apparently cheap one. The trained token horizon is the exception, because one run trains one horizon whichever quality surface turns out to be true: Section 0x02H fixes a single horizon per architecture from all the quality draws, then evaluates every draw at it.

The reviewable output is a decision record:

candidate registry and immutable artifact identities
training and prediction evidence inherited from M4.7/M4.8
measured runtime and quality evidence inherited from M5/M8
demand scenarios, prices, horizons, and service constraints
uncertainty propagation and discrete capacity calculations
expected cost plus tail-risk and regret comparisons
target-adjacent pilot evidence
proceed / revise / abort decision on the final training run

requirements handed downstream, not decided here
(M5 serving validation; M8.9/M8.10 for a low-bit deployment):
    canary evidence, release sequence, rollback trigger,
    and retained fallback artifact

An allocation with no pilot, no abort condition, or no stated rollback requirement is an extrapolation proposal, not a training decision. M4.9 states the release and rollback requirements its cost ledger assumed. M5 validates the serving system against them; for a low-bit deployment, M8.9 admits the artifact and M8.10 carries out its release and rollback. Section 0x02M describes that handoff.

0x00E Owns And Defers

M4.9 owns	Deferred owner
token multiplier, over-training definitions, and fixed-compute versus fixed-model comparisons	M4.7 owns the baseline training-compute optimum
validation-loss to downstream-metric mapping and apparent-emergence diagnostics	benchmark owners retain task semantics and raw evaluation outputs
lifecycle objective across training, prefill, decode, and demand, with release and rollback costs consumed as measured inputs	M4.1 owns analytic operation/resource derivation
uncertainty-aware discrete candidate comparison, pilot gates, and regret	M4.8 owns extrapolation support and prediction draws
consumption of measured latency, throughput, K/V, and capacity evidence	M5 owns serving mechanisms, scheduling, and cache state
consumption of qualified precision/artifact evidence; release and rollback requirements stated and handed downstream	M8 owns quantization, artifact construction, evaluation, release, and rollback mechanics

M4.9 may say that one measured runtime or artifact fails the lifecycle gate. It must not re-derive PagedAttention, scheduler policy, quantization transforms, or kernel compatibility to make that judgment.

0x00F Claim-Status Contract

Label	Meaning
Source-local	the linked paper or official source states the mechanism, equation, protocol, or result
Externally verified	another primary source or official implementation confirms the claim
Source-reported	a number belongs to the source’s model, workload, hardware, metric, and protocol; it is not a general constant
Rewrite derivation	the article derives the result from declared assumptions and units
Rewrite interpretation	a lifecycle, release, or engineering consequence synthesized from the cited evidence
Uncertain	support is insufficient; the claim becomes a scenario, sensitivity analysis, pilot question, or abort condition

Predicted cost is labeled as predicted. Measured runtime is labeled with its artifact, backend, hardware, workload, and software version. Source benchmarks remain source-reported until reproduced under the registered deployment contract.

0x00G Three-Article Reading Map

Article	Input	Output	It must not substitute for
M4.7 - Empirical Training Scaling Laws and Compute-Optimal Allocation	controlled runs and analytic compute definitions	measured and numerically certified baseline laws plus a training-compute allocation	hidden-scale evidence or deployment economics
M4.8 - Extrapolation, Alternative Scaling Laws, and Data Constraints	M4.7 estimators plus scale-ordered observations and data provenance	validated form, predictive support, data-constraint surface, uncertainty, or abort	lifecycle artifact selection
M4.9 - Over-Training, Downstream Metrics, and Lifecycle Allocation	M4.8 certified laws, frozen estimator, and prediction draws, plus measured downstream, serving, and artifact evidence	discrete train-and-serve recommendation, pilot gate, and training authorization or abort, with release and rollback requirements handed to M5 and, for a low-bit deployment, M8.9/M8.10	M5 runtime mechanisms or M8 artifact qualification

The conceptual sequence is:

M4.7: what relation was measured and numerically certified?
    -> M4.8: where does it predict, under which shape and data constraints?
    -> M4.9: which feasible artifact survives quality, demand, cost, and risk?

For an operational decision, revisit M4.9 after M5 and M8.10. Its equations need their measured runtime and qualified-artifact inputs; the article is not a replacement for those later modules.

0x00H Pinned Source-Version Contract

Over-training and lifecycle examples are unusually sensitive to paper revision: the training sweep, request assumptions, cost inputs, and caveats define the claim. This article audits its source-reported material against the versions below:

Source	Audited version	Contract consumed here
Hoffmann et al.	arXiv:2203.15556v1	training-compute reference frontier consumed from M4.7
Gadre et al.	arXiv:2403.08540v2	token-multiplier reparameterization, 104-model testbed, parallel loss curves, and aggregate task map
Wei et al.	arXiv:2206.07682v2	source definition and examples of claimed emergent abilities
Schaeffer et al.	arXiv:2304.15004v2	fixed-output metric transformation and apparent-emergence counterexample
Sardana et al.	arXiv:2401.00448v3	inference-aware objective, 47-model extreme-ratio sweep, and source-priced lifecycle examples

The 32x, 300x, 10,000 tokens/parameter, request-volume, MFU, sequence- length, and cost-saving values below are therefore examples under those source contracts. They are not defaults for a new workload. The article’s discrete enumeration, uncertainty propagation, and pilot rules, and the release and rollback requirements it hands downstream, are engineering interpretations that require local M5 and M8 measurements.

0x01 Over-Training, Downstream Metrics, And Apparent Emergence

Compute-optimal allocation is not always deployment-optimal allocation. A training-only frontier minimizes loss under one training budget. A product may instead accept a slightly worse training loss in exchange for a smaller model that is cheaper to serve repeatedly. That choice is often described as over-training: process more tokens per parameter than the training-only compute-optimal allocation would use.

The word creates three immediate risks:

risk 1:
    confusing a raw token-to-parameter ratio with a ratio relative to an optimum

risk 2:
    saying "32x over-trained" without saying whether N or C is held fixed

risk 3:
    treating smoother average benchmark behavior as a guarantee for every task

A fourth risk appears after evaluation. A benchmark score can contain a sharp threshold even when the model outputs feeding it change smoothly. The visible jump may be evidence about the metric rather than a newly formed internal mechanism. The converse is also important: showing that one metric can create a jump does not prove that all capability transitions are illusory.

This section therefore owns four linked contracts:

training coordinate:
    compute C, parameters N, processed tokens D, token multiplier D/N

over-training comparison:
    the reference ratio, the quantity held fixed, and the resulting loss change

downstream estimator:
    validation loss -> one registered aggregate task metric

metric-sensitivity audit:
    fixed model outputs scored by nonlinear, discontinuous, and graded metrics

It does not yet optimize training plus lifetime inference cost. Section 0x02 owns demand, prefill/decode work, device efficiency, monetary cost, and discrete deployment candidates. Here the purpose of over-training is to derive the performance surface and make evaluation claims auditable.

0x01A Token Multiplier Is Always D/N

Let

$$\rho=\frac{D}{N},$$

where:

Symbol	Meaning	Unit and contract
$N$	counted trainable model parameters for this scaling family	parameters; exclusions such as embeddings must be fixed
$D$	processed tokens that contribute to the training objective	tokens; not unique corpus size
$\rho$	token multiplier	processed tokens per counted parameter
$\rho^*$	compute-optimal token multiplier under a declared fit	same unit as $\rho$; distribution/recipe/compute specific
$m$	relative over-training factor $\rho/\rho^*$	dimensionless

The raw coordinate is always D/N. A relative claim needs a second number:

$$m=\frac{\rho}{\rho^*}.$$

Therefore:

rho = 640
    means 640 processed tokens per counted parameter

m = 32
    means rho is 32 times the declared reference rho*

if rho* = 20:
    rho = m rho* = 640

The source paper by Gadre et al. calls $M=D/N$ the token multiplier. This article uses $\rho$ so that $M$ is not confused with memory, model labels, or the relative factor $m$. The meaning is unchanged. One caption in the paper’s Figure 1 text reverses the written fraction while describing the ratio as tokens to parameters; the paper’s definition, equations, axes, and subsequent text consistently use $D/N$. The article follows the mathematical definition.

D in this ratio remains total processed training tokens. M4.8 Section 0x03 defines $D_{unique}$ and $D_{effective}$ for different questions; neither may be substituted into $\rho$:

rho = D_total / N

not:
    D_unique / N       # corpus support, not training work
    D_effective / N    # fitted performance equivalence, not processed work

0x01B Derive The Fixed-Compute Geometry Before Calling It Over-Training

Use the local training-compute approximation

$$C=kND,$$

where $k$ is the declared FLOP coefficient. Gadre et al. use $k=6$; M4.1 explains why an executable ledger may need a different coefficient. Combining $D=\rho N$ with the compute equation gives

$$C=k\rho N^2.$$

Solve for the model size:

$$N(C,\rho)=\sqrt{\frac{C}{k\rho}}.$$

Then recover tokens:

$$D(C,\rho)=\rho N =\sqrt{\frac{C\rho}{k}}.$$

Renderer-safe fallback:

rho = D / N
C = k * N * D = k * rho * N^2
N(C, rho) = sqrt(C / (k * rho))
D(C, rho) = sqrt(C * rho / k)

Now compare a reference allocation $(N^*,D^*)$ at ratio $\rho^*$ with a candidate whose ratio is $m\rho^*$ while holding training compute fixed. The equations imply

$$N_m=\frac{N^*}{\sqrt m}, \qquad D_m=D^*\sqrt m.$$

The new ratio is

$$\frac{D_m}{N_m} =m\frac{D^*}{N^*} =m\rho^*.$$

This square-root geometry is easy to lose in prose. A 32x ratio change at fixed compute does not process 32x as many tokens:

Relative factor $m$	$N_m/N^*$	$D_m/D^*$	$\rho_m/\rho^*$
1	1.000	1.000	1
4	0.500	2.000	4
16	0.250	4.000	16
32	0.177	5.657	32

At fixed model size, the comparison is different:

$$D_m=mD^*, \qquad C_m=mC^*.$$

The same 32x over-trained phrase can therefore describe either:

fixed N:
    same model, 32x tokens, 32x approximate training compute

fixed C:
    5.66x smaller model, 5.66x more tokens, same training compute

both:
    D/N is 32x the declared reference ratio

The distinction matters operationally. A fixed-model extension can reuse the architecture and continue training, but it consumes more training compute. A fixed-compute reallocation changes architecture size, optimizer state shape, parallelism, throughput, and eventual serving cost. They are not two names for one experiment.

0x01C Deriving The Parallel-Curve Law

Start from the joint reducible-loss model developed in M4.7 Sections 0x03 and 0x04:

$$L(N,D) =E+AN^{-\alpha}+BD^{-\beta}.$$

Gadre et al. first assume approximately equal model and data exponents:

$$\alpha=\beta.$$

Define

$$\eta=\frac{\alpha}{2}.$$

Substitute the fixed-compute coordinates from the previous section. The model term becomes

$$AN^{-\alpha} =A\left(\frac{C}{k\rho}\right)^{-\alpha/2} =Ak^{\eta}\rho^{\eta}C^{-\eta}.$$

The data term becomes

$$BD^{-\alpha} =B\left(\frac{C\rho}{k}\right)^{-\alpha/2} =Bk^{\eta}\rho^{-\eta}C^{-\eta}.$$

Let

$$a=Ak^{\eta}, \qquad b=Bk^{\eta}.$$

Then

$$L(C,\rho) =E+\left(a\rho^{\eta}+b\rho^{-\eta}\right)C^{-\eta}.$$

Renderer-safe fallback:

L(C, rho) = E + [a * rho^eta + b * rho^(-eta)] * C^(-eta)
eta = alpha / 2                  when alpha = beta
a = A * k^eta
b = B * k^eta

Define reducible loss

$$R(C,\rho)=L(C,\rho)-E.$$

Taking logs gives

$$\log R =\log\left(a\rho^\eta+b\rho^{-\eta}\right) -\eta\log C.$$

For every fixed $\rho$:

x-axis coefficient:
    slope = -eta

rho-dependent term:
    intercept = log(a rho^eta + b rho^-eta)

That is the parallel-curve result. The token multiplier moves the intercept; it does not move the log-compute slope under this equal-exponent model.

The compute-optimal ratio minimizes the intercept factor

$$g(\rho)=a\rho^\eta+b\rho^{-\eta}.$$

Differentiate:

$$g'(\rho) =a\eta\rho^{\eta-1} -b\eta\rho^{-\eta-1}.$$

Set the derivative to zero:

$$a\rho^{2\eta}=b.$$

Therefore

$$\rho^*=\left(\frac{b}{a}\right)^{1/(2\eta)}.$$

At the optimum, the model and data contributions are equal under the equal-exponent assumption:

$$a(\rho^*)^\eta =b(\rho^*)^{-\eta}.$$

Let $\rho=m\rho^*$. Divide the reducible loss at that ratio by the optimum at the same compute:

$$\frac{R(C,m\rho^*)}{R(C,\rho^*)} =\frac{m^\eta+m^{-\eta}}{2}.$$

This ratio is symmetric in $m$ and $1/m$. Moving toward too many tokens or too many parameters pays the same modeled reducible-loss penalty at reciprocal distance from the optimum. The result is a property of the separable, equal-exponent law, not a universal symmetry of training systems.

For a teaching value $\eta=0.1$ and $m=32$:

$$\frac{R(C,32\rho^*)}{R(C,\rho^*)} =\frac{32^{0.1}+32^{-0.1}}{2} \approx1.061.$$

The modeled reducible-loss penalty is about 6.1%, while the fixed-compute model is about 5.66x smaller. Total loss changes less than reducible loss when the floor $E$ is material. This shallow training-loss trade is why lifecycle cost can favor a deliberately non-optimal training allocation. It is not yet a lifecycle decision; Section 0x02 prices the saved inference work.

0x01D Unequal Exponents Need A Moving Reference

If $\alpha\ne\beta$, substituting a fixed raw ratio directly gives

$$L(C,\rho)=E +Ak^{\alpha/2}\rho^{\alpha/2}C^{-\alpha/2} +Bk^{\beta/2}\rho^{-\beta/2}C^{-\beta/2}.$$

The two terms now have different compute exponents. A constant raw $\rho$ does not produce an exactly single-slope curve over all compute. The source appendix obtains a parallel result by comparing a constant relative departure from the compute-optimal allocation, whose raw ratio can move with compute.

Under the compute constraint, the optimal coordinates scale as

$$N^*(C)\propto C^{\beta/(\alpha+\beta)}, \qquad D^*(C)\propto C^{\alpha/(\alpha+\beta)}.$$

Therefore the optimal raw token multiplier scales as

$$\rho^*(C)=\frac{D^*(C)}{N^*(C)} \propto C^{(\alpha-\beta)/(\alpha+\beta)}.$$

Only when $\alpha=\beta$ is $\rho^*$ constant in compute.

Now perturb the optimum at fixed compute:

$$N_m=\frac{N^*}{\sqrt m}, \qquad D_m=D^*\sqrt m.$$

The reducible loss becomes

$$R(C,m) =A m^{\alpha/2}(N^*)^{-\alpha} +B m^{-\beta/2}(D^*)^{-\beta}.$$

Both optimum terms scale with the same compute exponent:

$$(N^*)^{-\alpha}\propto C^{-\alpha\beta/(\alpha+\beta)},$$

and

$$(D^*)^{-\beta}\propto C^{-\alpha\beta/(\alpha+\beta)}.$$

So for fixed relative displacement $m$:

$$R(C,m) =h(m)C^{-\alpha\beta/(\alpha+\beta)},$$

where $h(m)$ absorbs fitted constants and the two $m$ factors. Parallelism is still predicted, but the statement has changed:

alpha = beta:
    fixed raw rho curves are parallel
    rho*(C) is constant

alpha != beta:
    fixed relative displacement m curves are parallel
    raw rho = m rho*(C) generally moves with compute

This distinction prevents a common implementation error: fit a constant $D/N$ grid, observe slight slope differences, and conclude that the appendix derivation failed. The experiment may not have held the quantity used by the derivation fixed.

0x01E What The Over-Training Paper Actually Tested

Gadre et al. built a source-reported testbed with:

104 trained models
approximately 0.011B to 6.9B parameters
three training distributions: C4, RedPajama, RefinedWeb
token multipliers M = D/N from 5 through 640
C4 evaluation loss as the default validation target
interpolation and extrapolation in both model size and token multiplier

The authors tuned base configurations near $\rho=20$, then reused those configuration policies at other multipliers rather than retuning every regime. They report that $\rho=5$ does not follow the same trend and hypothesize that regime-specific hyperparameter tuning may matter. Their fitted compute-optimal ratio also conflicts with empirical near-frontier observations in one appendix analysis. These are useful boundaries: 20 is not a universal optimum, and parallel lines are an empirical approximation over a tested region, not a theorem about every optimizer/data/model family.

Source: Gadre et al., Language Models Scale Reliably with Over-Training and on Downstream Tasks, Figure 2, licensed under CC BY 4.0. The desktop image retains the attributed source figure; the mobile image is an attributed responsive adaptation of its three-panel geometry. Curves, fit quality, and multiplier range are source-specific.

Figure 2 supplies empirical evidence for the reparameterization:

observed:
    roughly parallel reducible-loss trends over compute

modeled:
    common exponent eta, ratio-dependent intercept

not established universally:
    exact parallelism at every ratio, scale, corpus, or recipe

The paper’s main over-trained target was a source-reported 1.4B model trained on about 900B RedPajama tokens, approximately $\rho=640$. Relative to the paper’s near-optimal teaching reference $\rho=20$, that is a 32x ratio. The paper reports predicting its C4 evaluation loss from much cheaper fit runs. Those numbers demonstrate one extrapolation contract; they are not recommended production coordinates.

0x01F Loss-To-Average-Task Mapping Is A Second Estimator

Language-model validation loss is dense: nearly every evaluated target token contributes a real-valued observation. A benchmark accuracy can be sparse, discrete, prompt-sensitive, and close to chance at smaller scales. It is therefore unsafe to replace the loss target with a task score and assume the same fit cost, residual structure, or extrapolation support.

Gadre et al. use two fitted maps:

$$(C,\rho)\longmapsto \widehat L(C,\rho),$$

then

$$L\longmapsto \widehat{\operatorname{Err}}(L).$$

The downstream target is a uniform average of top-1 error over a registered task suite, not an unqualified notion of intelligence:

$$\overline{\operatorname{Err}} =\frac{1}{J}\sum_{j=1}^{J}\operatorname{Err}_j.$$

The paper proposes

$$\overline{\operatorname{Err}}(L) =\epsilon-k_qe^{-\gamma L},$$

where this section writes the fitted amplitude as $k_q$ to avoid confusing it with the training-FLOP coefficient $k$ in $C=kND$. The source paper uses $k$ for the task-map amplitude.

Variable contract:

Symbol	Meaning	Required registration
$L$	token-averaged cross-entropy on the chosen validation distribution	tokenizer, split, sequence weighting, units
$J$	number of registered evaluation tasks	task IDs and versions
$\operatorname{Err}_j$	top-1 error on task $j$	prompt, shots, answer parsing, baseline
$\overline{\operatorname{Err}}$	uniform average over those $J$ errors	weights fixed before target evaluation
$\epsilon,k_q,\gamma$	fitted task-map parameters	fit data, bounds, optimizer, uncertainty

Renderer-safe fallback:

Err_bar = (1 / J) * sum_j Err_j
Err_bar(L) = epsilon - k_q * exp(-gamma * L)

Why this form? Let perplexity be

$$\operatorname{PP}(L)=e^L.$$

Then

$$e^{-\gamma L} =\left(e^L\right)^{-\gamma} =\operatorname{PP}^{-\gamma}.$$

Therefore the same relation is a power law in perplexity:

$$\overline{\operatorname{Err}}(\operatorname{PP}) =\epsilon-k_q\operatorname{PP}^{-\gamma}.$$

The bounds also have an interpretation. As loss grows large, $e^{-\gamma L}$ tends to zero, so predicted error approaches $\epsilon$. As loss approaches the fitted irreducible floor $E$, the predicted lower limit is

$$\epsilon-k_qe^{-\gamma E}.$$

Those are fitted asymptotes, not logical limits on task performance. If $\epsilon$, $k_q$, and $\gamma$ are unconstrained, the curve can imply error outside [0,1] beyond support. Parameterize or reject such predictions rather than clipping them silently.

The local sensitivity to loss is

$$\frac{d\overline{\operatorname{Err}}}{dL} =k_q\gamma e^{-\gamma L}>0$$

when $k_q,\gamma>0$. A small loss prediction error $\delta L$ propagates as

$$\delta\overline{\operatorname{Err}} \approx k_q\gamma e^{-\gamma L}\delta L.$$

This derivative is only the conditional sensitivity with task-map parameters held fixed. A complete interval must also include uncertainty in $(\epsilon,k_q,\gamma)$, covariance with the loss fit if records are shared, task sampling, seeds, and scale extrapolation.

0x01G Chain The Maps Without Hiding Their Contracts

Combining the over-training law and task map gives

$$\widehat{\overline{\operatorname{Err}}}(C,\rho) =\widehat\epsilon -\widehat k_q \exp\left[-\widehat\gamma\widehat L(C,\rho)\right].$$

For the equal-exponent over-training form:

$$\widehat L(C,\rho) =\widehat E +\left( \widehat a\rho^{\widehat\eta} +\widehat b\rho^{-\widehat\eta} \right)C^{-\widehat\eta}.$$

The composition is conceptually simple but statistically nontrivial:

training runs
    -> fit loss surface theta_L = (E, a, b, eta)

evaluated checkpoints
    -> fit aggregate task map theta_Q = (epsilon, k_q, gamma)

candidate (C, rho)
    -> predict loss with support diagnostics
    -> map predicted loss to registered task average
    -> propagate both fits through the complete composition

A weak implementation fits the two stages once and plugs one point estimate into the other. A stronger implementation resamples the independent run and task units, refits both maps, and stores the joint prediction distribution:

def fit_loss_to_average_task(
    run_records,
    task_records,
    *,
    train_distribution_hash,
    validation_distribution_hash,
    suite_hash,
    task_weights,
):
    assert abs(sum(task_weights.values()) - 1.0) < 1e-12

    loss_fit = fit_overtraining_surface(
        run_records,
        train_distribution_hash=train_distribution_hash,
        validation_distribution_hash=validation_distribution_hash,
    )

    rows = aggregate_registered_tasks(
        task_records,
        suite_hash=suite_hash,
        weights=task_weights,
        keep_per_task=True,
    )
    task_fit = fit_bounded_loss_to_error(rows)

    return {
        "loss_fit": loss_fit,
        "task_fit": task_fit,
        "contract": {
            "train_distribution_hash": train_distribution_hash,
            "validation_distribution_hash": validation_distribution_hash,
            "suite_hash": suite_hash,
            "task_weights": task_weights,
        },
    }

Prediction must reject a changed contract rather than accepting a conveniently similar label:

def predict_registered_average(artifact, candidate, evaluation_contract):
    require_equal(
        evaluation_contract["validation_distribution_hash"],
        artifact["contract"]["validation_distribution_hash"],
    )
    require_equal(
        evaluation_contract["suite_hash"],
        artifact["contract"]["suite_hash"],
    )
    require_equal(
        evaluation_contract["task_weights"],
        artifact["contract"]["task_weights"],
    )

    support = artifact["loss_fit"].support_report(candidate)
    if not support["inside_validated_region"]:
        raise ExtrapolationError(support)

    loss_draws = artifact["loss_fit"].predict_draws(candidate)
    task_draws = artifact["task_fit"].predict_draws(loss_draws)

    return summarize_draws(task_draws), support

Do not pass only the mean and standard deviation of the first stage if the task map is nonlinear. In general,

$$\mathbb E[f(L)]\ne f(\mathbb E[L]).$$

Pass bootstrap or posterior draws through the second map, then summarize.

0x01H Source Evidence: Aggregate Behavior Is The Claim

Gadre et al. evaluated models trained on C4, RedPajama, and RefinedWeb, used C4 evaluation loss by default, and fit the average top-1 error of a source-defined 17-evaluation suite. A task entered that subset when at least one 0.154B model in the testbed performed at least 10 percentage points above random chance. The paper also reports ablations over a broader 46-evaluation suite.

Source: Gadre et al., Language Models Scale Reliably with Over-Training and on Downstream Tasks, Figure 3, licensed under CC BY 4.0. The desktop image retains the attributed source figure; the mobile image is an attributed responsive adaptation of its three-panel geometry. The fitted response is average top-1 error for the paper’s declared task suite, not a universal per-task law.

The source reports that average error was considerably more predictable than selected individual task errors. Its Table 2 includes individual relative prediction errors ranging from low single digits to very large values, while the registered average is much more stable in those examples. The paper also reports that removing its larger 1.4B, near-reference-ratio model from the task-map fit materially worsened one large-target prediction. Its approximate fit-compute ledger was about 100 A100 hours for the loss law and about 1,000 A100 hours for the downstream-error law under the source setup.

These are source-reported results, not universal budgets. They teach a decision rule:

dense loss target:
    often estimable from cheaper/smaller controlled runs

aggregate downstream target:
    may require larger models and more evaluation signal

individual task target:
    may need its own estimator, denser scale coverage, or may remain unreliable

0x01I Why Averaging Can Stabilize Without Making Every Task Predictable

Let the vector of task errors at one configuration be

$$\mathbf e=(e_1,\ldots,e_J)^\top.$$

The uniform average is

$$\bar e=\frac{1}{J}\mathbf 1^\top\mathbf e.$$

If the task-error covariance is $\Sigma$, then

$$\operatorname{Var}(\bar e) =\frac{1}{J^2}\mathbf 1^\top\Sigma\mathbf 1.$$

Under equal marginal variance $\sigma^2$ and common pairwise correlation $r$:

$$\operatorname{Var}(\bar e) =\frac{\sigma^2}{J}\left[1+(J-1)r\right].$$

This can be written using an effective task count

$$J_{eff} =\frac{J}{1+(J-1)r}.$$

If tasks were independent, $r=0$ and $J_{eff}=J$. If they move together, averaging helps less. This is a variance teaching model, not the paper’s downstream scaling equation. Real benchmark tasks have unequal variances, shared examples, related skills, common prompts, and correlated scoring failures, so estimate covariance by resampling the appropriate task/example hierarchy.

Aggregation can become smooth for several reasons:

noise cancellation:
    task-specific residuals partly average out

phase dispersion:
    different tasks improve at different scales, smoothing their mean

score density:
    many task-example outcomes create finer aggregate resolution

selection:
    excluding chance-level tasks can increase visible signal

The last item is also a policy choice. If the suite is selected after looking at all target-model results, the reported predictability is conditional on that selection. Freeze the selection rule using pilot roles and repeat it inside bootstrap/model-selection loops. Never tune the suite on the final hidden-scale target and then call it a held-out prediction.

Weighted aggregation changes the estimand:

$$\bar e_w=\sum_{j=1}^{J}w_je_j, \qquad \sum_jw_j=1.$$

Uniform task weights, equal example weights, traffic weights, domain weights, and worst-group objectives answer different questions. Register one before fitting. A production target may legitimately differ from the source’s uniform 17-task mean, but it then requires a new validation exercise.

0x01J Individual Tasks And OOD Distributions Are Separate Targets

The source paper explicitly identifies two boundaries.

First, individual downstream-task prediction is noisy. A task can sit near chance, use a discontinuous score, have few examples, depend strongly on prompting, or improve along a different latent direction than average language modeling loss. Lower aggregate loss does not guarantee monotonic improvement on every task.

Second, out-of-distribution loss extrapolation can fail. The paper trains on English-filtered C4 and reports high relative prediction errors in some evaluations involving programming-language data and Penn Treebank formatting, while one German C4 evaluation remained more predictable. CommonCrawl overlap between nominally different datasets also complicates a simple ID/OOD label.

The correct claim hierarchy is:

Claim	Evidence needed	Invalid shortcut
validation loss on registered domain	held-out scale and ratio predictions on that domain	training loss or a related corpus name
average error on registered suite	frozen tasks/weights plus hidden-scale aggregate test	one smooth plot using all scales
one task’s error	task-specific signal and extrapolation audit	aggregate predictability
OOD loss or task error	explicit target distribution and OOD holdout	in-distribution residuals
in-the-wild product quality	traffic-weighted evaluation and feedback contract	academic benchmark mean
safety capability	dedicated high-resolution evaluation and uncertainty	lower perplexity

When one scalar loss is used to compare models trained on different data distributions, equal loss need not imply equal knowledge, calibration, toxicity, domain performance, or task ranking. The task map is conditional on the training and validation distributions that generated it.

An artifact should therefore preserve distribution and evaluation lineage:

loss_to_task_contract:
  training_data:
    snapshot_hash: "..."
    mixture_hash: "..."
    tokenizer_hash: "..."
  validation_loss:
    dataset_hash: "..."
    aggregation: "mean_nats_per_nonpadding_target_token"
  downstream_suite:
    suite_hash: "..."
    task_versions: ["..."]
    prompt_template_hashes: ["..."]
    shots_by_task: {"...": 0}
    answer_parser_hash: "..."
    task_weights: "uniform_over_registered_tasks"
  scale_support:
    N_range: ["...", "..."]
    rho_range: ["...", "..."]
    C_range: ["...", "..."]
  estimator:
    loss_family: "equal_exponent_overtraining"
    task_family: "bounded_exponential_in_loss"
    grouped_resampling: "recipe/config/seed + task/example"

0x01K End-To-End Bootstrap For The Two-Stage Map

The independent units differ by stage. Training seeds and configurations govern the loss fit. Task definitions and examples govern downstream aggregation. If the same checkpoints feed both stages, their errors are also coupled. A practical bootstrap can preserve these relationships:

def bootstrap_loss_to_task(
    run_groups,
    task_manifest,
    candidate,
    n_boot,
    rng,
):
    draws = []

    for _ in range(n_boot):
        sampled_runs = resample_complete_run_groups(run_groups, rng)

        # Recreate all run-level observations and refit the over-training law.
        loss_rows = rebuild_loss_observations(sampled_runs)
        loss_fit = fit_overtraining_surface(loss_rows)

        # Resample tasks and examples according to the declared estimand.
        sampled_suite = resample_registered_suite(
            task_manifest,
            rng=rng,
            preserve_task_weights=True,
            cluster_examples_by_source=True,
        )
        task_rows = evaluate_or_reaggregate(sampled_runs, sampled_suite)
        task_fit = fit_bounded_loss_to_error(task_rows)

        # The nonlinear map receives the complete predicted-loss draw.
        predicted_loss = loss_fit.predict(candidate)
        predicted_error = task_fit.predict(predicted_loss)

        draws.append({
            "loss": predicted_loss,
            "average_error": predicted_error,
            "loss_parameters": loss_fit.parameters,
            "task_parameters": task_fit.parameters,
            "support": loss_fit.support_report(candidate),
        })

    return summarize_joint_draws(draws)

If task examples cannot be rerun for every bootstrap, cache item-level model outputs and rescore them. Do not cache only aggregate accuracy; that prevents metric sensitivity, parser corrections, cluster resampling, and continuous companion scores later.

The final report should include at least:

loss prediction:
    median, interval, source-scale distance, residual diagnostics

aggregate task prediction:
    median, interval, suite definition, task covariance diagnostic

individual tasks:
    observed/predicted residuals, but no guarantee inherited from the mean

OOD statement:
    target distribution, evidence status, and explicit refusal if unsupported

fit budget:
    training compute and evaluation cost used to build both stages

0x01L “Emergence” Can Name An Observation Or A Mechanism

Wei et al. define an emergent ability as one not present in smaller models but present in larger models, such that it cannot be predicted by simply extrapolating smaller-model task performance. That is an observational definition over a chosen task, metric, model family, and scale axis.

It does not by itself identify why the curve changes. Several objects can produce a visible threshold:

model behavior:
    a representation, circuit, algorithm, or composition effect changes

evaluation geometry:
    a nonlinear metric magnifies smooth changes

decision geometry:
    a continuous score crosses a discrete threshold

measurement resolution:
    small expected success remains below the test set's observable granularity

experimental protocol:
    prompt, tokenizer, data, training recipe, or model family changes with scale

sampling sparsity:
    too few scale points make a steep smooth curve look discontinuous

Call the first a mechanism claim and the measured jump a metric claim. Evidence that a metric curve jumps establishes the second. It does not isolate the first without controls.

Schaeffer et al. ask a particularly clean diagnostic question: for fixed model outputs on a fixed task and model family, can changing only the metric create or remove the apparent transition? Their answer for the examined cases is yes. That result is strong evidence about metric sensitivity. The paper itself states that it should not be interpreted as proving that large language models cannot display emergent abilities.

0x01M From Cross-Entropy To Target-Token Probability

For one observed target token $v^*$ at scale $s$, cross-entropy is

$$\ell(s)=-\log \widehat p_s(v^*).$$

Therefore the probability assigned to that target token is

$$\widehat p_s(v^*)=e^{-\ell(s)}.$$

If loss changes smoothly with scale, the assigned target probability also changes smoothly because the exponential is continuous.

There are two precision warnings.

First, exponentiating a token-averaged cross-entropy gives the geometric mean target-token probability:

$$e^{-\bar\ell} =\left(\prod_{i=1}^{m}p_i\right)^{1/m},$$

not the arithmetic mean probability and not automatically the probability that greedy decoding selects the correct token.

Second, the probability assigned to a full target sequence under teacher forcing is

$$p(y_{1:m}\mid x) =\prod_{i=1}^{m} p(y_i\mid x,y_{<i}).$$

This product is also the probability of sampling that exact sequence from the same autoregressive conditionals. Greedy exact-match accuracy is a different, deterministic event involving argmax choices and exposure to generated rather than target prefixes after an error.

Schaeffer et al.’s teaching model compresses these differences into one common per-token correctness probability $p(s)$ and assumes the required events can be multiplied. The article keeps that approximation because it exposes metric geometry, but does not promote it to a universal generation model.

0x01N Derive The Nonlinear Exact-Match Curve

Suppose a task requires $m$ output tokens and each required correctness event has common probability $p(s)$. Under the source’s simplifying independence approximation:

$$A_{exact}(s)\approx p(s)^m.$$

Renderer-safe fallback:

exact_sequence_accuracy(scale) ~= p_correct_token(scale) ** target_length

The derivative is

$$\frac{dA_{exact}}{dp}=mp^{m-1}.$$

More revealing is log elasticity:

$$\frac{d\log A_{exact}}{d\log p}=m.$$

A one-percent relative change in the common token probability produces an approximately $m$-percent relative change in exact-match probability. Sequence length is a gain applied by the metric.

For $m=20$:

Common token probability $p$	Exact match $p^{20}$	Expected token errors $20(1-p)$
0.80	0.0115	4.0
0.85	0.0388	3.0
0.90	0.1216	2.0
0.95	0.3585	1.0
0.98	0.6676	0.4

Both columns are smooth in $p$. The first nevertheless appears threshold-like on a linear accuracy axis because multiplication suppresses partial progress. The second preserves graded information.

An approximately linear companion metric is the expected count of incorrect tokens:

$$\mathbb E[K_{wrong}] \approx m(1-p).$$

Token edit distance is not exactly this expression because insertions, deletions, variable output length, and dependent errors matter. The source uses it as an approximately linear metric whose behavior makes partial sequence progress visible.

The important word is appears. $p(s)^m$ is continuous whenever $p(s)$ is continuous. A plot can look abrupt without containing a mathematical discontinuity.

0x01O Finite Test Resolution Can Hide The Smooth Tail

Suppose true exact-match probability is $A$ and the test set contains $n$ independent examples. Empirical accuracy has resolution

$$\Delta\widehat A=\frac{1}{n}.$$

The probability of observing no successes is

$$P(K=0)=(1-A)^n.$$

The probability of observing at least one is

$$P(K\ge1)=1-(1-A)^n.$$

To have at least probability $q$ of observing one success, require

$$n\ge\frac{\log(1-q)}{\log(1-A)}.$$

For small $A$, use $\log(1-A)\approx-A$:

$$n\gtrsim\frac{-\log(1-q)}{A}.$$

At $q=0.95$:

$$n\gtrsim\frac{3}{A}.$$

If true exact-match probability is 0.001, roughly 3,000 independent examples are needed for a 95% chance of observing even one success. With a 100-example test, several smaller models can all report zero, then a larger model can produce the first visible nonzero point. The empirical curve jumps because the instrument was coarse.

Independence is again a simplification. Template clusters, repeated examples, contamination, and correlated failure reduce effective sample size. Use the cluster-aware machinery from M4.8 0x03 rather than treating nominal example count as resolution.

Better statistics do not mean generating more prompts after seeing where the jump occurs. The resolution plan, stopping rule, and scale grid must be declared before the final emergence test or repeated inside a valid sequential design.

0x01P Discontinuous Metrics Create Actual Score Jumps

For a multiple-choice item, let $z_c(s)$ be the score of the correct option and $z_j(s)$ the scores of alternatives. Define the correct-option margin

$$\Delta(s)=z_c(s)-\max_{j\ne c}z_j(s).$$

The ordinary grade is

$$G(s)=\mathbf 1[\Delta(s)>0].$$

Even if every option score and the margin change smoothly, $G$ jumps from zero to one when the margin crosses zero. Averaging over a finite test set creates a staircase with increments 1/n.

A continuous companion can expose progress before the crossing. If $q_j$ is the normalized option probability and $y_j$ the one-hot target, the Brier score is

$$B=\sum_j(q_j-y_j)^2.$$

Log score, target-option probability, calibrated margin, and Brier score answer different questions from top-1 grade, but each retains information discarded by the threshold. They should accompany the operational metric, not necessarily replace it. Exact match and top-1 grade can still be the product requirements; the continuous companion is the scientific instrument for detecting smooth pre-threshold change.

A task-level threshold can add another discontinuity:

$$H(s)=\mathbf 1[\widehat A(s)\ge\tau].$$

This answers whether a release bar was crossed. It cannot reveal the shape of the underlying capability curve. Plot and model $\widehat A(s)$ with an interval, then overlay $\tau$ as a decision threshold rather than fitting a scaling law to $H$.

0x01Q Source Evidence: Hold Outputs Fixed, Change The Metric

Schaeffer et al. begin from smoothly decreasing per-token cross-entropy and smoothly increasing assigned correct-token probability. Their Figure 2 then applies four score families:

nonlinear:
    exact sequence accuracy

discontinuous:
    multiple-choice grade

approximately linear:
    token edit distance

continuous:
    Brier score

The unchanged source artwork remains available as Schaeffer et al., Figure 2. It is not served locally because the paper’s CC BY-NC-ND 4.0 terms prohibit derivatives and restrict reuse to noncommercial contexts. The m4-7-metric-created-threshold diagram above is an independently designed article explanation: it derives the metric transformations used in this text, does not reproduce the paper’s panel layout or data, and is not new empirical evidence.

The source analyzes fixed outputs and selected metrics. Its result is a metric-sensitivity explanation for examined claims, not proof that all emergence is impossible.

The paper tests the explanation on InstructGPT/GPT-3 arithmetic tasks, performs a BIG-Bench metric meta-analysis, and constructs apparent transitions in vision tasks by choosing thresholded metrics. It reports that changing metrics or increasing statistical resolution removes the examined apparent jumps.

The strongest defensible article statement is:

Nonlinear and discontinuous metrics, finite resolution, and sparse scale sampling can create apparently sharp and unpredictable benchmark curves from smoothly changing model outputs. Every emergence claim should therefore pass a metric-sensitivity and resolution audit.

The article must not strengthen this into:

wrong:
    all emergent abilities are fake

wrong:
    continuous metrics are always the true task metric

wrong:
    smooth output scores prove no internal mechanism changed

wrong:
    a real product threshold is scientifically meaningless

Operational thresholds can matter while their crossing remains predictable from a continuous score. Internal mechanisms can change even when aggregate metrics are smooth. Conversely, a benchmark jump can occur without a new mechanism. These are different axes of evidence.

0x01R A Metric-Sensitivity Protocol

The clean audit keeps examples and outputs fixed. Re-running generation after changing prompts, temperatures, parsers, or model checkpoints mixes metric sensitivity with a new experiment.

def metric_sensitivity_report(cached_outputs, task_contract):
    require_hash(cached_outputs.examples, task_contract.example_hash)
    require_hash(cached_outputs.prompts, task_contract.prompt_hash)
    require_equal(cached_outputs.decoding, task_contract.decoding)

    rows = []
    for scale_id, outputs in group_by_scale(cached_outputs):
        exact = exact_sequence_match(outputs)
        edit = token_edit_distance(outputs)

        if outputs.has_choice_probabilities:
            grade = top1_choice_grade(outputs)
            brier = brier_score(outputs)
            margin = correct_choice_margin(outputs)
        else:
            grade = brier = margin = None

        rows.append({
            "scale_id": scale_id,
            "exact_match": cluster_interval(exact),
            "token_edit": cluster_interval(edit),
            "choice_grade": cluster_interval(grade),
            "brier": cluster_interval(brier),
            "margin": cluster_interval(margin),
            "target_length": summarize_target_length(outputs),
        })

    return {
        "curves": rows,
        "resolution": metric_resolution(rows),
        "change_points": compare_smooth_and_broken_forms(rows),
        "multiple_testing": registered_family_report(task_contract),
    }

Required views:

View	What it tests	Failure it can expose
exact match vs token edit	nonlinear sequence conjunction	partial progress hidden by all-or-nothing score
top-1 grade vs probability/margin/Brier	discontinuous decision	smooth confidence movement hidden before rank crossing
score by target length	metric gain $m$	longer targets create sharper curves
raw successes and interval	finite resolution	zero plateau below 1/n
dense local scale points	curve continuity	sparse sampling makes steep smooth section look abrupt
same task across model families	family dependence	architecture/recipe change mislabeled universal threshold
registered tasks vs all searched tasks	multiplicity	most surprising curve selected post hoc

Do not choose the continuous metric solely because it removes a jump. Declare why each score is scientifically and operationally meaningful, then report the set. A continuous proxy with no connection to the product objective can be as misleading as an all-or-nothing grade with no diagnostic resolution.

0x01S Paper-By-Contract Comparison

Paper	Independent coordinate	Primary response	Main contribution here	Boundary to preserve
Hoffmann et al.	$N,D,C$	language-model loss	joint loss and compute-optimal reference	source coefficients and accounting are setup-specific
Gadre et al.	compute and $D/N$	validation loss; average top-1 error	over-trained reparameterization and two-stage task map	individual tasks and OOD settings can be unreliable
Wei et al.	model scale across families	task metric curves	observational definition and examples of emergent abilities	visible task curve does not identify mechanism
Schaeffer et al.	fixed outputs, changed metric	nonlinear, discontinuous, linear, continuous scores	metric-created apparent thresholds and resolution tests	does not prove all emergence is illusory
BNSL	arbitrary scale variable	arbitrary measured response	smooth broken forms can represent changing slopes	a fitted break is not automatically a capability transition

The papers answer different questions. BNSL can detect a statistical break; Schaeffer can show that a metric transformation creates one; Wei catalogs observational task curves; Gadre predicts one average task response from loss. No single result substitutes for the others.

0x01T Training And Evaluation Integration

Over-training and downstream prediction require training records that expose actual work and evaluation records that preserve raw outputs.

Training state

per checkpoint:
    counted parameter policy and exact N
    processed loss tokens D_total
    unique-data and exposure diagnostics from M4.8 0x03
    rho = D_total / N
    training FLOPs from measured or declared ledger
    optimizer/schedule horizon and current schedule position
    data mixture, tokenizer, code, and recipe hashes

Do not infer $D$ from checkpoint name. Gradient accumulation, packed-sequence loss masks, restarts, dropped batches, curriculum mixtures, and data-loader replay can move actual contributing-token count.

An over-trained checkpoint from a long schedule is not equivalent to the same token horizon taken from a schedule designed to end there. Learning-rate area, warmup fraction, decay position, optimizer moments, and validation selection differ. Either train schedule-matched runs for each target multiplier or label checkpoint extrapolation as a separate estimator.

Evaluation state

per item:
    task/version, example ID, source cluster
    full prompt and prompt-template hash
    target and target length
    generated token IDs and decoded output
    parser result and parse-failure reason
    option logits/probabilities where available
    exact match, token edit, grade, Brier/margin companion scores

Caching item-level outputs permits metric changes without changing model behavior. Caching only aggregate accuracy makes Schaeffer-style controls impossible and hides parser failures.

Fit roles

loss-fit runs:
    estimate (C, rho) -> validation loss

task-map runs:
    estimate loss -> registered average task error

metric-audit outputs:
    fixed generations scored several ways

hidden-scale target:
    untouched until both estimator families and metrics are frozen

The source paper reports that downstream-error prediction benefited from a larger fit model than loss prediction. Budgeting only tiny pilot models can therefore make the second stage underpowered even when the first looks stable. Fit-budget allocation is itself an experiment.

Runtime serving cost does not enter $\rho$ or the task map. Smaller parameter count may reduce weight memory and token latency, but batching, quantization, KV cache, prefill/decode split, and hardware utilization determine realized cost. Section 0x02 joins this performance model to those measured ledgers.

0x01U Design Choices And Their Boundaries

Choice	Useful when	Hidden assumption	Required sensitivity
call $D/N$ the multiplier	comparing token-heavy training regimes	same parameter and token conventions	recount both axes under alternatives
normalize by $\rho^*$	discussing degree of over-training	reference optimum is identified	frontier interval and ratio range
assume $\alpha=\beta$	source-like near-equal exponents	fixed raw-ratio curves share slope	unequal-exponent derivation
share one $\eta$ across ratios	parallel trends are supported	no ratio-by-compute interaction	fit ratio-specific slopes and residuals
fit average top-1 error	product target is that exact suite mean	task set/weights remain fixed	per-task and weighted aggregates
select high-signal tasks	avoiding chance-dominated noise	selection is not target-contaminated	repeat selection inside fit roles
use validation loss as mediator	loss captures relevant progress	equal loss implies comparable task behavior	train-data and domain interactions
exact match as requirement	whole output must be correct	partial progress is not decision-relevant	token-level companion metric
Brier or margin companion	probabilities are available/calibrated	option set and scoring are stable	calibration and prompt variants
broken fit on task score	residuals indicate a slope change	break is identifiable before target	smooth alternatives and hidden scales
dense scale sampling	local transition is affordable	other protocol variables stay fixed	family/recipe controls

0x01V Over-Training And Metric Failure Lab

Failure	Symptom	Why the conclusion fails	Response
token multiplier written $N/D$	larger number appears to mean less data	ratio is inverted	define and assert rho = D_total / N
32x stated without reference	readers infer 32x tokens or compute	held-fixed quantity is missing	report $\rho,\rho^*,m,N,D,C$
unique tokens used in $\rho$	repeated run seems cheap	corpus support replaces processed work	use $D_{total}$
fixed-$N$ and fixed-$C$ curves mixed	square-root geometry seems inconsistent	they are different experiments	separate ledgers and labels
equal-exponent law used after unequal fit	fixed raw-ratio curves drift	common-slope premise changed	moving-optimum derivation
$\rho^*=20$ universalized	every model is labeled over/under-trained identically	optimum is contract-specific	local frontier interval
parallel lines inferred by eye	ratio interaction is missed	slopes/intercepts have uncertainty	hierarchical fit and residual test
task map fit to final target	prediction appears exact	target leaked into estimator	hidden-scale task holdout
average task law applied per task	individual predictions fail	aggregation smoothness was transferred	task-specific report/refusal
average reweighted after results	target becomes easier post hoc	estimand changed	immutable suite weights
ID loss map used for OOD task	domain failure surprises deployment	source support was crossed	target-domain pilot
exact match called capability	zero then jump looks mechanistic	metric hides partial progress	token edit/probability companion
token independence presented as fact	$p^m$ overclaims sequence behavior	errors and decoding prefixes are dependent	label teaching approximation
geometric-mean probability called accuracy	exponentiated loss is misinterpreted	probability, argmax, and sample event differ	preserve each definition
finite zero treated as true zero	small models seem incapable	test resolution is too coarse	binomial/cluster interval and more data
top-1 grade modeled as continuous	threshold crossing surprises fit	score discards margin	store probabilities and margin
continuous metric chosen post hoc	jump is removed by design	metric selection is outcome-driven	preregister metric family
one apparent jump proves mechanism	paper narrative outruns evidence	protocol/metric alternatives remain	mechanism-specific experiment
Schaeffer generalized universally	all emergence dismissed	paper scope is fixed outputs/examined claims	preserve explicit boundary
benchmark search multiplicity ignored	one dramatic curve is highlighted	winner’s curse	registered family and multiplicity control

0x01W Over-Training And Downstream-Metric Review Gate

Before accepting an over-training or downstream claim, answer:

• Is the token multiplier defined consistently as $\rho=D_{total}/N$?
• Are $\rho^*$ and $m=\rho/\rho^*$ separate from the raw ratio?
• Does every Nx over-trained statement name its reference and held-fixed quantity?
• Are fixed-model and fixed-compute comparisons kept separate?
• Is the square-root reallocation derived under fixed compute?
• Is the parallel-curve law derived from the joint loss rather than asserted?
• Is exact fixed-$\rho$ parallelism scoped to $\alpha=\beta$?
• For unequal exponents, is the fixed relative displacement around the moving optimum used?
• Are source model sizes, token multipliers, data distributions, and reported extrapolation errors labeled source-reported?
• Is validation loss predicted before being mapped to a downstream target?
• Is the downstream target one exact, frozen aggregate with declared weights?
• Are task selection, prompts, shots, parsers, baselines, and task versions hashed?
• Is uncertainty propagated through both fitted maps rather than through point estimates only?
• Are average-task claims kept separate from individual-task claims?
• Are OOD distributions treated as new prediction targets?
• Are item-level outputs retained for metric sensitivity and resampling?
• Is $p^m$ labeled as a sequence-metric teaching approximation with dependent error and decoding caveats?
• Are exact match and top-1 grade accompanied by graded/continuous diagnostics?
• Is finite test resolution quantified rather than treating observed zero as exact zero capability?
• Are metric, scale-grid, and multiple-testing choices frozen before the final target?
• Does the article avoid claiming that all emergence is illusory?

This performance analysis adds two bridges to the scaling workflow:

training allocation bridge:
    (N, D, C) -> D/N -> over-trained loss surface

evaluation bridge:
    validation loss -> registered task average -> metric-sensitivity audit

Neither bridge prices deployment. Section 0x02 combines a fixed-quality performance constraint with training cost, expected inference demand, prefill/decode ledgers, device efficiency, and a target-adjacent pilot. The current section supplies the quality surface and prevents the lifecycle optimizer from treating a fragile benchmark jump as a stable product target.

0x02 Lifecycle Allocation And The Decision Pipeline

The compute-optimal question from M4.7 Section 0x04 fixes a training budget and asks which model/token pair minimizes loss. Deployment reverses the decision. A product team commonly has a quality floor, several executable model configurations, and an uncertain number of future requests. It must choose the artifact before that demand is observed.

Given a frozen quality target, a validated loss surface, a finite set of trainable and deployable configurations, and uncertain lifetime demand, which model should be trained, for how many tokens, and under what pilot gate?

This is not the claim that serving cost can be predicted from parameter count alone. M4.1 owns the detailed FLOP/byte ledger. M5 owns prefill, decode, K/V state, batching, and serving measurements. M8, especially M8.10, owns the quantized artifact, its quality validation, release, and rollback. M4.7 owns the baseline training-compute fit. M4.9 owns the decision that consumes those measurements together with the fitted quality surface.

The lifecycle bridge is therefore:

validated quality surface and bootstrap draws
    + frozen quality target
    + demand scenarios
    + measured training/prefill/decode cost profiles
    + discrete trainable/deployable candidates
    -> lifecycle cost and regret distributions
    -> target-adjacent pilot
    -> proceed, revise, or abort

0x02A Reverse The Training-Only Optimization

Let the fitted pretraining-loss surface be

$$L(N,D_{tr}) = E+\frac{A}{N^\alpha}+\frac{B}{D_{tr}^\beta},$$

where $N$ is the counted model size and $D_{tr}$ is processed pretraining tokens. In the training-only problem, compute $C_{tr}$ is fixed and loss is minimized. In the lifecycle problem, quality is fixed at a declared target $\ell$ and total cost is minimized:

$$(N^*,D_{tr}^*) = \underset{N,D_{tr}:L(N,D_{tr})\le\ell}{\arg\min} \left[ J_{tr}(N,D_{tr})+J_{pf}(N;W)+J_{dec}(N;W) \right].$$

$W$ is a frozen workload contract rather than one token count. It includes request volume, prompt and output-length distributions, concurrency/SLO class, serving artifact, hardware, precision, and software version. The three cost terms are training, prefill, and decode. Costs can be dollars, accelerator seconds, joules, or another additive unit, but one optimization must not mix units.

Renderer-safe fallback:

choose (N, D_train) minimizing
    training_cost
  + lifetime_prefill_cost
  + lifetime_decode_cost
subject to predicted_quality meeting the frozen target

The inequality matters. Rounding a continuous token solution upward may make a candidate slightly better than the target. Equality is useful for derivation; the executable decision uses <= ell and rechecks it after rounding.

This is a fixed-quality comparison. Serving a smaller model that is also worse is not a lifecycle efficiency result. If pretraining loss is only a proxy for product quality, the registered loss-to-task map and its uncertainty from 0x01 become part of the constraint.

0x02B Eliminate The Token Axis At Fixed Quality

The quality constraint turns a two-dimensional continuous search into a one-dimensional curve. Rearrange

$$\ell-E-\frac{A}{N^\alpha} = \frac{B}{D_{tr}^\beta}.$$

Define the residual data allowance

$$R_N(\ell) = \ell-E-A N^{-\alpha}.$$

When $R_N(\ell)>0$, the tokens required by this fitted law are

$$D_{req}(N,\ell) = \left( \frac{B}{R_N(\ell)} \right)^{1/\beta}.$$

Renderer-safe fallback:

R_N = target_loss - E - A * N**(-alpha)
D_required = (B / R_N)**(1 / beta)   only when R_N > 0

The feasibility boundary is explicit:

$$N> \left( \frac{A}{\ell-E} \right)^{1/\alpha}.$$

This requires $\ell>E$. If $\ell\le E$, the fitted surface has no finite model/token pair at or below the target because $E$ is its asymptotic floor.

Below that model size, this law predicts that no finite token budget reaches the target. That is a model-dependent extrapolation statement, not proof of a fundamental capability threshold. A broken law, architecture change, better data, distillation, or post-training can invalidate it.

Differentiate the required-token curve:

$$\frac{dD_{req}}{dN} = -D_{req} \frac{\alpha A N^{-\alpha-1}} {\beta R_N(\ell)}.$$

The derivative is negative: within this fitted family, a larger model needs fewer training tokens to hit the same loss. The lifecycle decision asks whether the saved training work is worth carrying the larger model through every future prefill and decode.

Under Sardana et al.’s coarse FLOP approximation,

$$J_{FLOP}(N) = 6N D_{req}(N,\ell)+2N D_{inf},$$

where $D_{inf}$ is lifetime input plus output tokens. Its derivative is

$$\frac{dJ_{FLOP}}{dN} = 6D_{req} \left[ 1- \frac{\alpha A N^{-\alpha}} {\beta R_N(\ell)} \right] +2D_{inf}.$$

With zero inference demand, this reduces to the training-only tangent from M4.7 Section 0x04. Positive demand adds a penalty proportional to $N$, shifting the stationary point toward a smaller model and more training tokens. The paper solves the resulting fixed-quality problem numerically rather than claiming a general closed form.

0x02C Source Evidence: Extra Training Can Repay Inference

Sardana et al. make the trade concrete with a source-reported schematic. Their example compares a Chinchilla-style 13B model with an equal-quality 7B model trained longer. The 7B option spends an additional $0.7\times10^{22}$ pretraining FLOPs but saves $2.4\times10^{22}$ inference FLOPs over 2T inference tokens, for a net source-reported saving of $1.7\times10^{22}$ FLOPs (17%). Those numbers are one paper scenario, not a universal 7B-versus-13B exchange rate.

Article-authored reconstruction of Sardana et al., Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws, Figure 1. It preserves the source-reported 7B versus 13B equal-quality example, the 2T inference-token scenario, and the displayed FLOP totals, but it does not reproduce the paper artwork or exact plot coordinates and is not a new measurement.

The same paper trained 47 MPT-family models from 150M to 6B parameters at ratios from 10 to as high as 10,000 tokens per parameter. It reported continued improvement within the tested ranges, but also found that fitting only ordinary-ratio runs overestimated gains in the extreme long-training regime. Lifecycle optimization therefore cannot extrapolate a training-only fit thousands of tokens per parameter and call the recommendation validated. The long-ratio region must appear in the fit or in a hidden target-adjacent test.

The source’s optimization is deliberately data-rich and compute-constrained. It assumes sufficient fresh data for the longer run and, conditioned on fixed pretraining loss, the same inference demand for different model sizes. If unique data is capped, the repetition-aware surface from M4.8 0x03 replaces the abundant-data token term. If latency, price, or accessibility changes adoption, demand must become candidate-dependent rather than being copied unchanged between models.

0x02D Derive The Break-Even Demand

For two equal-quality candidates, write lifecycle cost as

$$J_i(Q)=T_i+Q I_i,$$

where $T_i$ is one-time training cost, $I_i$ is measured variable inference cost per request (or per declared workload unit), and $Q$ is lifetime demand in that same unit. Suppose candidate S is smaller but trained longer, so $T_S>T_L$ and $I_S<I_L$. Set their lifecycle costs equal:

$$Q_{break} = \frac{T_S-T_L}{I_L-I_S}.$$

Renderer-safe fallback:

Q_break = (training_cost_small - training_cost_large)
        / (inference_cost_large - inference_cost_small)

The numerator is the training premium. The denominator is the saving on each future workload unit. The ratio is meaningful only when:

• both candidates meet the same frozen quality target;
• training and inference costs use one additive unit;
• the per-request workload distributions are comparable;
• the denominator is positive;
• fixed migration, storage, and engineering costs are either shared or included separately.

Using only the coarse source FLOPs from Figure 1,

$$Q_{break} = \frac{5.2\times10^{22}-4.5\times10^{22}} {2(13-7)\times10^9} \approx 5.83\times10^{11} \text{ inference tokens}.$$

At 2T tokens, the source’s smaller model is beyond that break-even point. In a real system, replace 2N with measured prefill and decode costs for the exact artifact. A quantized 7B artifact and a BF16 13B artifact do not share one cost-per-FLOP coefficient, and one-token decode can be bandwidth-limited even when its FLOP count looks small.

0x02E FLOPs Are Work; They Are Not A Price

The simplified objective treats one training FLOP, one prompt FLOP, and one decode FLOP as interchangeable. They are not. Sardana et al. therefore extend their objective with phase-specific utilization and cost-per-FLOP factors. A more explicit ledger begins with the effective price of realized arithmetic:

$$\kappa_{h,p,\phi} = \frac{r_h} {3600\,P_{peak}(h,p)\,u_{h,p,\phi}},$$

where:

Symbol	Meaning
$h$	hardware/profile identifier
$p$	precision and artifact profile
$\phi$	phase: training, prefill, or decode
$r_h$	accelerator price in currency per hour
$P_{peak}(h,p)$	declared peak arithmetic rate at that precision
$u_{h,p,\phi}$	measured realized-FLOP fraction for that phase
$\kappa$	currency per realized algorithmic FLOP under the profile

Then a phase-separated variable-cost model is

$$J(x,s) = \kappa_{tr}F_{tr}(x) +\kappa_{pf}F_{pf}(x,s) +\kappa_{dec}F_{dec}(x,s) +J_{fixed}(x,s).$$

Renderer-safe fallback:

lifecycle_cost =
    train_flops  * dollars_per_train_flop
  + prefill_flops * dollars_per_prefill_flop
  + decode_flops  * dollars_per_decode_flop
  + declared fixed costs

u is not licensed to absorb every serving effect. If the phase is dominated by bytes, queueing, or a latency constraint, use a measured accelerator-second or request-cost profile directly. The generalized implementation contract is:

$$J(x,s) = J_{tr}^{measured}(x) +Q_s\, \mathbb E_{w\sim\mathcal W_s} \left[ J_{pf}^{measured}(x,w) +J_{dec}^{measured}(x,w) \right] +J_{fixed}(x,s).$$

The expectation is over a versioned workload trace, not an unqualified “average request.”

Sardana et al.’s real-cost example used source-reported assumptions of 50% training MFU, 50% input/prefill MFU, 1% generation MFU, 70 input tokens, 215 output tokens, A100-80GB for training, A100-40GB for inference, and an INT8 inference artifact. Under only the utilization weights, one representative request contributes

prefill effective-token term = 70  / 0.50 =    140
decode  effective-token term = 215 / 0.01 = 21,500

before multiplying by the inference cost coefficient and 2N. That teaching calculation explains why equal FLOP counts can have very different real costs. It does not assert that modern decode universally runs at 1% MFU.

The source also states that its real-cost equation omits latency and treats MFU and cost per FLOP as independent of model size, configuration, and sequence length. The measured workload-bin contract below is an engineering extension that relaxes those assumptions; it is not claimed as Sardana et al.’s estimator.

0x02F The Runtime Contract Is An Input, Not A Scaling Law

For every deployable candidate $x$, freeze a runtime profile:

candidate identity:
    architecture hash
    parameter convention and exact N
    checkpoint / training recipe
    serving artifact hash and precision

hardware/software:
    accelerator and topology
    runtime, kernel, compiler, and driver versions
    parallelism and batch policy identifier

workload bins:
    prompt-length bin
    output-length bin
    concurrency / SLO class

measurements per bin:
    accepted requests
    accelerator-seconds or currency
    TTFT and TPOT distributions
    peak memory and KV capacity
    prefill/decode throughput
    quality result on the exact artifact

This article does not derive those measurements from a scheduler model. M5 supplies the phase benchmark and feasibility surface. M8 supplies and validates the exact precision artifact. If either profile changes, the lifecycle optimization is rerun; the old optimum does not survive by definition.

Three shortcuts are unsafe:

Shortcut	Hidden assumption	Safer contract
inference FLOPs = 2N per token	sequence length, attention, K/V traffic, batching, and vocabulary head are negligible	use M4.1 estimate for screening, then measured phase profiles
one average MFU	training, prefill, and decode expose equal arithmetic intensity	record phase- and workload-bin-specific realized cost
smaller model is deployable	memory fit implies runtime and SLO support	require exact artifact benchmark and SLO feasibility

Precision belongs in the candidate identity because it changes throughput, memory, and sometimes quality. This section does not teach how INT8/INT4 artifacts are created. It consumes M8’s signed result: artifact a passed its quality and compatibility gate under runtime profile r.

0x02G Demand Is A Scenario Distribution

The model is selected before lifetime demand is known. One point forecast hides the variable most likely to reverse the decision. Define scenario $s$ by

$$s= \left( Q_s, \mathcal W_s, h_s, p_s, r_s, \pi_s \right),$$

where $Q_s$ is request volume, $\mathcal W_s$ is the prompt/output workload distribution, $h_s/p_s/r_s$ identify hardware, precision, and price profiles, and $\pi_s$ is an optional scenario weight.

A minimal report includes:

low demand:
    conservative adoption, short lifetime, or high cache reuse

base demand:
    registered product forecast and workload trace

high demand:
    successful adoption, longer lifetime, or more generated tokens/request

stress:
    decode-heavy mix, lower utilization, price change, or tighter SLO

For candidate $x$, scenario regret is

$$R_s(x) = J_s(x)-\min_{z\in\mathcal F_s}J_s(z).$$

Alternative decision policies answer different questions:

$$x_{EV} = \arg\min_x\sum_s\pi_sJ_s(x),$$ $$x_{robust} = \arg\min_x\max_s R_s(x),$$

or a declared upper-tail criterion such as CVaR. Do not quietly choose the policy after seeing which candidate it favors. Expected value is suitable only when scenario probabilities and tail tolerance are defensible.

0x02H Continuous Optima Do Not Train Or Deploy

The analytic solution may request 9.73B parameters and 2.418T tokens. The system can execute only configurations that satisfy architecture, optimizer, data, hardware, and runtime constraints. Let a candidate be

$$x_j= (a_j,N_j,D_j,g_j,t_j,v_j),$$

where $a_j$ is an architecture configuration, $g_j$ a training topology and precision profile, $t_j$ a serving artifact/runtime profile, and $v_j$ a versioned data/evaluation contract.

The feasible set should check at least:

Constraint	Example executable condition
architecture	integer layers/heads/width; divisibility under tensor and pipeline parallelism
training state	parameters, gradients, optimizer state, activations, and buffers fit the topology
token horizon	whole global batches/optimizer steps; data policy and unique-token cap respected
wall clock	measured step time and failure allowance fit the reservation window
serving artifact	exact precision/layout supported by the pinned runtime and kernels
runtime memory	weights, K/V state, allocator reserve, and batch buffers fit
SLO	measured TTFT/TPOT/throughput satisfies the scenario class
quality	artifact-level quality constraint holds with declared confidence

For one architecture, each bootstrap quality-surface draw $q$ first answers its own question: how many tokens would this draw need? Solving the fixed-quality equation of 0x02B under $\theta^{(q)}$ gives the per-draw requirement

$$D_{req,j}^{(q)} = D_{req}(N_j,\ell;\theta^{(q)}),$$

set to $+\infty$ when the draw’s residual allowance $R_{N_j}(\ell)$ is not positive: under that draw, no finite horizon reaches the target at this model size.

These requirements are not yet a decision. One run is trained, with one horizon $D_j$, and its learning-rate schedule is built for that horizon before launch (Section 0x01T). The draws describe which quality surface might be true; the run cannot choose its horizon after learning which one is. A useful quality gate is therefore a chance constraint on that single horizon, evaluated under every draw:

$$\Pr_q \left[ L(N_j,D_j;\theta^{(q)})\le\ell \right] \ge 1-\delta.$$

This probability describes uncertainty under the declared grouped-bootstrap procedure. It is not a universal probability that the final run succeeds; it does not include unmodeled distribution shifts or a new architecture family.

A five-draw teaching example shows which single horizon satisfies this constraint. Take $N_j=7\times10^9$ and target $\ell=2.00$, and hold $A$, $\alpha$, $B$, and $\beta$ at the source-reported Hoffmann et al. fit quoted in M4.7 (406.4, 0.34, 410.7, 0.28). Let only the floor $E$ move between draws, and set $\delta=0.2$: the trained horizon must meet the target under at least four of the five draws, so at most one may miss. Horizons are rounded up to whole optimizer steps of $2^{22}$ tokens. These are illustrative draws, not a fit: a real grouped bootstrap moves every coefficient together and uses hundreds of draws.

Draw $q$	Floor $E^{(q)}$	Requirement $D_{req,j}^{(q)}$	Meets $\ell$ if the median requirement (3.39T) is trained	Loss at the chance-constrained $D_j$ (6.24T)
1	1.66	1.59T	yes	1.950
2	1.68	2.59T	yes	1.970
3	1.69	3.39T	yes	1.980
4	1.71	6.24T	no	2.000
5	1.83	$+\infty$	no	2.120

A horizon meets a draw’s target once it reaches that draw’s requirement, because loss falls as tokens are added. Sorted, the requirements are 1.59T, 2.59T, 3.39T, 6.24T, and $+\infty$, and the smallest horizon that reaches four of them is the fourth, 6.24T. Training that horizon meets the target under draws 1-4 and misses it under draw 5: $4/5=0.8\ge1-\delta$. Training the median requirement instead, 3.39T, meets it under only three draws, $3/5=0.6$, which fails; the median hides exactly this tail. The same rounded $D_j$ is also the horizon checked against the unique-token cap and the wall-clock window in the table above; checking a median horizon there can pass a run whose actual horizon exceeds the data budget.

The general rule rests on that monotonicity. In the fitted law of 0x02A,

$$\frac{\partial L}{\partial D_{tr}} = -\beta B D_{tr}^{-\beta-1} <0 \qquad \text{whenever } B>0,\ \beta>0,$$

so within one draw the target is met exactly when the horizon reaches that draw’s requirement:

$$L(N_j,D;\theta^{(q)})\le\ell \iff D\ge D_{req,j}^{(q)}.$$

The fraction of draws that meet the target at horizon $D$ is therefore the fraction of requirements at or below $D$. With $n_q$ draws, at most $\lfloor\delta n_q\rfloor$ may miss, so the smallest admissible horizon is the $k_\delta$-th smallest requirement, rounded up to an executable step:

$$D_j = \operatorname{ceil\_to\_step} \left[D_{req,j}^{[k_\delta]}\right], \qquad k_\delta=n_q-\lfloor\delta n_q\rfloor,$$

where $D_{req,j}^{[i]}$ denotes the $i$-th smallest of the $n_q$ requirements. $n_q$ counts every bootstrap replicate. A replicate whose refit fails stays in the count as an unreachable requirement ($+\infty$): dropping it would shrink $n_q$ and quietly loosen the constraint. In the example, $k_\delta=5-1=4$; with $n_q=200$ and $\delta=0.05$, $k_\delta=200-10=190$, so the horizon is the 190th smallest requirement. If that order statistic is $+\infty$, more than $\lfloor\delta n_q\rfloor$ draws cannot reach the target at any horizon, and the architecture fails the quality gate. Rounding up only adds tokens, so the rounded horizon still satisfies the constraint; because rounding preserves order, rounding every requirement first gives the same $D_j$. Use the order statistic itself rather than an interpolated quantile. Interpolation can land between two requirements and leave fewer than $k_\delta$ draws satisfied, and it returns $+\infty$ or NaN when the interpolation bracket touches an unreachable draw.

Renderer-safe fallback:

required[q] = D_req(N_j, target; theta_q)    # +inf if draw q cannot reach target
k = n - floor(delta * n)                     # exact rational arithmetic (see 0x02J)
D_j = ceil_to_step(sorted(required)[k - 1])  # reject N_j if this is +inf
evaluate every draw q at the same D_j

A tempting shortcut gives each draw its own horizon, $\operatorname{ceil\_to\_step}[D_{req,j}^{(q)}]$, and evaluates each draw at its own value. Every reachable draw then meets the target by construction. Here draws 1-4 pass, draw 5 is rejected, and the reported probability is $4/5=0.8$. The number happens to match the correct one, but it measures something else: the fraction of draws under which the target is reachable at all. No run trains four horizons, and if every draw had been reachable, the shortcut would report probability 1 whichever single horizon was trained. The shortcut also prices the wrong run. Averaging $6ND$ over the four per-draw horizons gives $1.45\times10^{23}$ training FLOPs, while the one run that satisfies the gate needs $6N_jD_j=2.62\times10^{23}$, 1.8 times more. Those extra tokens are the price of the $1-\delta$ confidence, and per-draw records never charge it.

The order-statistic solution needs loss to be non-increasing in tokens within every draw. The separable law above satisfies that, and so does a repetition-aware surface whose repeated tokens only saturate (M4.8 0x03): there, extra epochs can stop helping, which makes some requirements $+\infty$, but they do not raise loss. A certified broken form fitted through a non-monotonic segment (M4.8 0x02), or a model of repeated-subset damage (M4.8 0x03), can rise with tokens. Then compute the fraction of draws meeting the target directly at each executable horizon and keep the smallest horizon that passes; per-draw horizons are wrong there too.

0x02I Propagate Three Sources Of Decision Uncertainty

Lifecycle allocation contains at least three different uncertainty objects:

quality-surface uncertainty:
    grouped bootstrap draw theta^(q)

demand uncertainty:
    scenario s or sampled workload trace

runtime uncertainty:
    repeated phase benchmark / price / availability draw r

For candidate $x_j$, the quality draws have already done their work in Section 0x02H: they fixed its one horizon $D_j$ and decided whether it passes the chance constraint. Because the run trains $D_j$ whichever surface is true, a quality draw no longer changes the candidate’s cost. Under a tail draw the target is missed, which the constraint already allows with probability at most $\delta$, so the candidate is not dropped from the cost comparison for that draw. Cost and deployability vary with demand and runtime, so evaluate

$$J_{s,r}(x_j)$$

only when the candidate remains feasible under that demand and runtime draw. Report:

Output	Why it matters
quality-feasibility probability	fraction of draws meeting the target at the one trained horizon $D_j$; catches candidates that meet target only under the median fit
winner probability	shows whether one candidate consistently minimizes the declared objective across demand and runtime draws
lifecycle-cost quantiles	exposes demand and runtime tails
scenario-specific regret	separates a cheap median from an expensive wrong-demand decision
probability of SLO/memory infeasibility	keeps economic optimum inside the deployable set
break-even distribution	shows whether the demand forecast is meaningfully on one side of the crossover

Do not multiply three point estimates and call the result uncertainty-aware. Keep correlations where evidence provides them: long outputs can lower decode throughput; price and hardware availability can move together; the spread of the fit draws sets each architecture’s chance-constrained horizon, so a wider spread can change both required tokens and the candidate ranking.

Functional-form uncertainty remains outside one bootstrap. Repeat the entire decision under every extrapolation-certified law from M4.8 Sections 0x01 and 0x02. If the candidate changes across plausible laws, that disagreement belongs in the pilot design rather than being averaged away.

0x02J A Discrete Lifecycle Solver

The implementation should enumerate executable candidates, solve every quality-surface draw’s required horizon, and fix one chance-constrained horizon per architecture before pricing it. It should not optimize a fractional width and round only after declaring a winner, and it should not let each draw train its own horizon.

from fractions import Fraction
from math import floor, inf
from statistics import mean


def lifecycle_candidates(
    fit_draws,
    quality_target,
    demand_scenarios,
    architecture_catalog,
    runtime_profiles,
    delta,
):
    records = []
    n = len(fit_draws)
    k = draws_that_must_pass(n, delta)  # k_delta, an exact integer

    for architecture in architecture_catalog:
        N = counted_parameters(architecture)

        # Each draw's own requirement; None means unreachable at this N.
        # Contract: a returned horizon D satisfies
        # predict_loss(theta, N, D) <= quality_target in floating point,
        # e.g. the upper end of a bracketing root solve, not its midpoint.
        # A failed-refit draw returns None here and +inf from predict_loss,
        # so it always counts as a miss.
        required = [
            tokens_required_for_loss(theta, N, quality_target)
            for theta in fit_draws
        ]
        D_cont = chance_constrained_horizon(required, k)
        if D_cont is None:
            records.append(infeasible(
                architecture, "quality_chance_constraint",
                reachable_fraction=mean(d is not None for d in required),
            ))
            continue

        # One trained horizon for this architecture, shared by every draw.
        D = ceil_to_complete_optimizer_step(D_cont, architecture)
        draw_losses = [predict_loss(theta, N, D) for theta in fit_draws]
        draws_met = sum(loss <= quality_target for loss in draw_losses)
        # Holds by the contract above, since rounding up only adds tokens;
        # compare counts, not float probabilities.
        assert draws_met >= k
        quality_probability = draws_met / n
        train_profile = build_or_lookup_train_profile(architecture, D)

        for artifact in runtime_profiles.compatible_artifacts(architecture):
            if not artifact_passed_quality_gate(artifact):
                continue

            for scenario in demand_scenarios:
                serving = runtime_profiles.lookup(artifact, scenario.workload_id)
                checks = check_all_constraints(
                    architecture=architecture,
                    tokens=D,
                    train_profile=train_profile,
                    serving_profile=serving,
                    scenario=scenario,
                )

                if not checks.feasible:
                    records.append(infeasible(
                        architecture, checks.reason, scenario.id
                    ))
                    continue

                cost = (
                    train_profile.variable_cost
                    + scenario.requests * serving.mean_request_cost
                    + declared_fixed_costs(artifact, scenario)
                )
                records.append(feasible_record(
                    architecture=architecture,
                    artifact=artifact,
                    scenario=scenario.id,
                    tokens=D,
                    predicted_losses=draw_losses,
                    quality_probability=quality_probability,
                    lifecycle_cost=cost,
                ))

    summary = aggregate_by_candidate(records)
    summary = add_scenario_regret_and_winner_probability(summary)
    return choose_by_preregistered_decision_policy(summary), summary


def draws_that_must_pass(n, delta):
    # k_delta = n - floor(delta * n), computed from the decimal delta.
    risk = Fraction(str(delta))
    if not 0 <= risk < 1:
        raise ValueError("delta must lie in [0, 1)")
    if n == 0:
        raise ValueError("no fit draws: the chance constraint is undefined")
    return n - floor(risk * n)


def chance_constrained_horizon(required, k):
    # Valid when loss is non-increasing in D under every draw (0x02H).
    ordered = sorted(inf if d is None else d for d in required)
    return None if ordered[k - 1] == inf else ordered[k - 1]

The listing counts draws instead of comparing probabilities, because binary floating point cannot hold most decimal risk levels exactly. With $\delta=0.18$ and 50 draws, $k_\delta=41$, and 41 of 50 is exactly the 82% the gate asks for, yet 1 - 0.18 evaluates to 0.8200000000000001, so the comparison 41 / 50 >= 1 - 0.18 is false and a horizon that meets the constraint fails the check. The floor has the same problem: 0.29 * 100 evaluates to 28.999999999999996, so floor(delta * n) allows 28 misses instead of 29, and the solver would choose the 72nd smallest of 100 requirements instead of the 71st. That error is on the safe side, but the result is no longer the smallest admissible horizon. Computing $k_\delta$ once with exact rational arithmetic and comparing integer counts keeps the code equal to the formula. An empty draw set is rejected explicitly: with no draws, the constraint says nothing about the surface. A risk level outside $[0,1)$ is rejected too: only that range puts $k_\delta$ between $1$ and $n_q$. A negative $\delta$ gives $k_\delta>n_q$, and $\delta\ge1$ gives $k_\delta\le0$. Without the check, a negative $\delta$ or a $\delta\ge2$ fails later with an IndexError that does not name the cause, but for $1\le\delta<2$ Python’s negative indexing hides the error: ordered[k - 1] wraps around to the order statistic that $\delta-1$ would choose. $\delta=1$ gives $k_\delta=0$, and ordered[-1] silently picks the largest requirement. $\delta=1.2$ gives $k_\delta=n_q-\lfloor1.2\,n_q\rfloor=-\lfloor0.2\,n_q\rfloor$, which depends on the number of draws: with 50 draws, $k_\delta=-10$, and ordered[-11] returns the 40th smallest requirement, the one $\delta=0.2$ would select, so an invalid risk level produces a plausible-looking horizon.

The assert relies on the contract stated for tokens_required_for_loss. A bisection that returns the midpoint of its last bracket can stop just short of the root. If the root lies just above a step boundary and the midpoint just below it, rounding up lands on that boundary, still short of the root: the $k_\delta$-th draw misses the target by a tiny margin, and the assert stops a horizon one step shorter than the formula’s. Returning the upper end of the bracket, where the loss is already at or below the target, removes the case.

The solver needs an audit trail. Every rejection records its reason. Every cost record points to the fit-draw set and the per-draw losses at its rounded horizon, the architecture hash, workload scenario, runtime benchmark, price snapshot, and artifact hash. A spreadsheet with silently overwritten MFU cells is not a reproducible lifecycle study.

0x02K Worked Decision Ledger

The source-reported Figure 1 comparison can be rewritten as an auditable break-even table:

Candidate	Parameters	Training FLOPs	Inference FLOPs/token	Lifecycle FLOPs at 2T tokens
smaller, longer-trained	7B	$5.2\times10^{22}$	$1.4\times10^{10}$	$8.0\times10^{22}$
larger, Chinchilla-style	13B	$4.5\times10^{22}$	$2.6\times10^{10}$	$9.7\times10^{22}$

The comparison establishes a mechanism, not a deployment recommendation. A real decision table should add:

quality:
    predicted loss interval
    exact artifact task gate

training:
    rounded token horizon
    accelerator-hours, price, and wall-clock interval

prefill:
    workload bins, measured request cost, TTFT feasibility

decode:
    workload bins, measured request cost, TPOT feasibility

uncertainty:
    low/base/high demand costs
    winner probability and regret quantiles
    break-even interval

Suppose demand lies below the break-even interval in the low scenario, crosses it in the base scenario, and lies above it in the high scenario. The correct report is not “the 7B wins.” It is:

training-only / low demand:
    larger candidate retains the lower lifecycle cost

base demand:
    decision is sensitive to demand and runtime uncertainty

high demand:
    smaller longer-trained candidate has lower lifecycle cost

decision:
    chosen under the declared expected-value, robust-regret, or tail policy

0x02L The Target-Adjacent Pilot Is A Frozen Test

The lifecycle optimizer combines several extrapolations. Before the full run, train a configuration close enough to test the actual decision boundary. A target-adjacent pilot is not merely the largest run the team can afford. It is chosen to discriminate among plausible decision outcomes.

Good pilot placements include:

• the selected architecture at a predeclared fraction of its final token horizon, with loss checkpoints predicted before launch;
• the next smaller or larger executable architecture where candidate rankings disagree;
• an extreme token-ratio point needed by the selected lifecycle candidate but absent from the fitting grid;
• the exact deployment artifact on a representative workload trace, when runtime cost rather than quality is the unstable axis.

Before launch, freeze:

fit and candidate-law version
bootstrap draws and prediction intervals
quality target and downstream map
candidate catalog and rounding rule
demand scenarios and decision policy
runtime/artifact benchmark versions
pilot checkpoints and allowed peeks
pass, abort, and refit thresholds

At milestone $k$, compare observed loss $L_k$ with the frozen predictive mean $\widehat L_k$ and predictive standard deviation $\sigma_{pred,k}$:

$$z_k = \frac{L_k-\widehat L_k}{\sigma_{pred,k}}.$$

Use prediction uncertainty, not only coefficient uncertainty. Consecutive checkpoints are correlated, so |z| > 1.96 at any arbitrary peek is not a valid repeated-testing rule. Predeclare milestone checkpoints and either use a simultaneous envelope, an alpha-spending rule, or a simple conservative gate.

One concrete gate is:

PASS only if all hold:
    final pilot loss is inside the frozen prediction interval
    no predeclared sequential drift rule fired
    projected final training cost remains within tolerance
    selected artifact passes quality, memory, and SLO checks
    candidate remains acceptable under the frozen regret policy

ABORT / REOPEN if any hold:
    loss trajectory breaches the one-sided abort envelope
    long-ratio slope is outside the validated family
    measured phase cost moves break-even beyond the supported demand range
    rounded final horizon violates data, time, memory, or budget constraints
    exact artifact fails quality or runtime feasibility

An abort is not permission to delete the miss. Preserve the original forecast, the pilot result, and the triggered rule. If the pilot is used to modify the loss law, demand model, candidate set, or runtime profile, it becomes development data. Refit, freeze a new study version, and reserve a new untouched pilot before claiming validation.

0x02M The End-To-End Decision Pipeline

The complete study now has four frozen boundaries:

1. Measurement freeze: define N, D, loss, run lineage, data, and compute.
2. Estimator freeze: fit candidate laws on lower-scale groups, choose among them on the next larger selection groups, then score the frozen choice once on still larger final hidden-scale groups (M4.8 0x01A).
3. Decision freeze: register quality, demand scenarios, runtime profiles, candidate set, uncertainty propagation, and selection policy.
4. Pilot freeze: register target-adjacent observations and pass/abort rules.

def scaling_study_training_authorization(study):
    manifest = validate_measurement_contract(study.run_manifest)
    fit_groups, selection_groups, final_groups = scale_ordered_split(manifest)

    # Fit every candidate law on the lower-scale groups only.
    candidate_laws = fit_and_diagnose_all_laws(fit_groups)

    # Choose the law set on the next larger groups (M4.8 0x01A).
    selected_laws = select_on_validation_scale(candidate_laws, selection_groups)
    if not selected_laws:
        return abort("no candidate law passed selection-scale validation")
    frozen_laws = refit_if_predeclared(
        selected_laws,
        fit_groups + selection_groups,
    )

    # Open the final hidden scale once. A miss aborts; it never prunes the set.
    final_report = score_frozen_laws_once(frozen_laws, final_groups)
    archive_final_score(final_report)  # kept on a pass and on a miss
    if not final_report.passes_predeclared_gate:
        return abort("no extrapolation-certified quality surface")
    certified_laws = frozen_laws

    # Functional-form uncertainty stays outside the bootstrap (0x02I):
    # bootstrap and solve the decision once per certified law.
    decisions_by_law = {}
    for law in certified_laws:
        # Resample groups within each development role and refit this law
        # in every draw. The law fixes its form, floor policy, transition
        # family, and break count. Inside a draw, the frozen rule may only
        # re-choose settings that keep that form, such as the crop point or
        # the lower cutoff. A replicate whose refit fails is returned as a
        # draw with an unreachable requirement, never dropped.
        # Unlike M4.8 0x01F, a draw never switches forms:
        # switching would pool forms inside one order statistic, weighted
        # by draw counts rather than evidence (see the text below).
        # final_groups are never resampled, reselected, or refit.
        fit_draws = grouped_bootstrap_complete_pipeline(
            fit_groups,
            selection_groups,
            law,
        )
        decisions_by_law[law.id] = lifecycle_candidates(
            fit_draws=fit_draws,
            quality_target=study.quality_target,
            demand_scenarios=study.demand_scenarios,
            architecture_catalog=study.architecture_catalog,
            runtime_profiles=study.runtime_profiles,
            delta=study.quality_risk,
        )

    # Draws from different laws are never pooled into one horizon. If the
    # laws choose different candidates, the ledger keeps every law's
    # decision and the disagreement sets the pilot placement (0x02L).
    decision, ledger = reconcile_across_laws(
        decisions_by_law,
        study.decision_policy,
    )

    frozen = freeze_decision_contract(decision, ledger, study)
    pilot = run_target_adjacent_pilot(frozen.pilot_spec)

    if not evaluate_frozen_pilot_gate(pilot, frozen):
        archive_prediction_and_miss(frozen, pilot)
        return reopen_as_new_study_version(frozen, pilot)

    return authorize_final_run(decision, frozen, pilot)

Why three groups rather than two? Suppose one candidate law misses the hidden scale by a true 3.5%, the tolerance is 3%, and scoring on a finite set of groups adds Gaussian noise with a standard deviation of one percentage point. If the same hidden groups both choose the surviving laws and certify them, this law survives whenever its score happens to land under 3%, which is about 31% of the time, and on those occasions it is certified with a reported error that averages about 2.4%. The certificate is the same lucky score that kept the law in the set. With separate selection groups the law can still slip through selection, but the final groups give it a fresh score centered on 3.5%, so the frozen gate rejects it about 69% of the time. It is now certified only when two independent scores both land under 3%: about 9.5% of the time ($0.3085^2$) instead of about 31%. The fresh score is also an honest measurement, but only when it is reported whatever the gate decides: over all outcomes it averages the true 3.5%. Reported only on a pass, it is truncated in the same way and again averages about 2.4%. The final score is therefore published with its interval on a miss as well as on a pass, just as Section 0x02L preserves a pilot miss. These are teaching numbers from a truncated-normal calculation, not a source result.

The split follows M4.8 0x01A and the fit-role rule of Section 0x01T: the hidden-scale target stays untouched until the estimator families are frozen. The final groups are opened once. If the frozen law set misses the predeclared gate there, the study aborts or reopens with a new hidden scale. Dropping only the failing laws and keeping the rest would turn the final groups into a second selection set, and nothing untouched would remain to certify the survivors.

The bootstrap then runs once per certified law, as Section 0x02I requires. Pooling the draws of two laws into one lifecycle_candidates call would make the $k_\delta$-th smallest requirement an order statistic of a mixture: if one law’s requirements all lie above the other’s, the horizon lands in one law’s range or the other’s depending on how many draws each law contributed, a weight set by draw counts rather than by any evidence about which law is right. Solved per law, each horizon reflects one law’s fit uncertainty, and a disagreement between the laws’ decisions stays visible for the pilot to test.

The same mixture would form inside a single law’s bootstrap if a draw could switch forms, as the selection-path bootstrap of M4.8 0x01F allows. Each form’s share of the draws would then be how often it wins a resampled selection. That frequency measures how stable the choice of form is under resampling, not how likely the form is to be the true law, yet here it would decide which form’s tail the $\delta$ budget may discard. M4.8 reports those selection frequencies to expose structural uncertainty; here the form stays fixed within each law’s bootstrap, and the whole decision is repeated per certified form.

The break count and the floor policy belong to that form, although the selection-path bootstrap of M4.8 0x01F reselects them in every draw with the frozen rule. A zero-break and a one-break BNSL fit can match the same development rows and still extrapolate differently, so letting the break count float inside one law’s bootstrap would rebuild the mixture there. If laws with different break counts or floor policies are all certified, each is a separate law with its own bootstrap and decision. Only the choices that leave the extrapolating form unchanged, such as the crop point or the lower cutoff, are rerun in every draw.

For the chance constraint, M4.8 supplies the certified law set, the frozen selection rule, and the run/configuration groups and design strata to resample. The draws the constraint uses are generated from these once per certified law, with the form held fixed, like the M4.7 0x07E draws for one declared form; grouped_bootstrap_complete_pipeline in the listing above produces them. M4.8’s own 0x01F prediction draws can switch forms, so they and their selection frequencies are read as evidence of how stable the choice of form is, never as the draws over which the $\delta$ budget is spent.

The final authorization is a training decision, not a deployment release. M5 still owns serving-system validation. For a low-bit deployment, M8.9 admits the actual quantized artifact/runtime representation and M8.10 owns its complete release tuple, observability and rollback. M4.9 does not add a second production release controller. Its handoff has four immutable objects:

M4.9 recommendation receipt:
    study/pilot identity, selected candidate, frozen quality/demand assumptions,
    runtime-profile IDs, uncertainty ledger and training authorization

actual trained artifact:
    resulting checkpoint/config/data/run identity; not the predicted artifact

M8.9 low-bit intake/admission:
    transformed artifact, graph, loader/kernel/cache ABI and parity evidence

M8.10 deployment result:
    exact served tuple, workload, quality/resource gates and release verdict

Each downstream receipt references its predecessors without rewriting them. If measured training or deployment results change the lifecycle decision, a new M4.9 study version consumes those observations with new frozen assumptions and any required untouched validation. The original recommendation remains historical evidence; a release failure cannot be erased by retrospectively changing its study inputs. Generic nonquantized production ownership is a separate program boundary, not implied by this M8 intake.

0x02N Alternative Objectives And Design Choices

The fixed-quality lifecycle objective is not the only legitimate formulation. Choose one before solving:

Decision contract	Objective	Appropriate when	Main danger
fixed quality	minimize lifecycle cost subject to $L\le\ell$	quality requirement is non-negotiable	proxy loss may not equal product quality
fixed lifecycle budget	minimize loss subject to $J\le B$	total spend is hard-capped	budget uncertainty can make boundary brittle
expected cost	minimize $\sum_s\pi_sJ_s$	scenario probabilities are credible	rare costly demand is underweighted
robust regret	minimize worst scenario regret	policy must tolerate misspecified demand	can overpay for implausible extremes
portfolio/staged deployment	train or retain several artifacts	routing tiers justify fixed complexity	extra training, storage, and operational burden

Other choices must remain visible:

• continuous root solve vs discrete enumeration: use continuous analysis for intuition, discrete enumeration for the decision;
• analytic FLOPs vs measured phase costs: use analytic ledgers for screening and instrumentation checks, measured costs for final runtime ranking;
• one demand point vs scenarios: report break-even and sensitivity even if one base forecast drives procurement;
• mean fit vs conservative quality gate: a mean-quality equality is not a high-confidence quality guarantee;
• one law vs certified law set: disagreement between plausible extrapolation forms is decision uncertainty, not optimizer noise;
• single pilot vs sequential program: repeated pilots require an explicit adaptive design and new untouched final evidence.

0x02O Failure Lab

Failure	Attractive but wrong conclusion	What actually failed	Repair
training-only optimum shipped	minimum training FLOPs means minimum product cost	lifetime inference was omitted	optimize fixed-quality lifecycle objective
training and inference FLOPs priced equally	one FLOP has one universal dollar cost	phase utilization, precision, hardware, and bytes differ	phase-specific measured cost profiles
input and output tokens merged	all inference tokens have equal runtime	prefill and sequential decode expose different kernels/utilization	separate prompt and generation ledgers
median demand only	one candidate is clearly cheapest	break-even lies inside forecast uncertainty	low/base/high scenarios and regret
average request only	sequence distribution cannot change ranking	tails alter attention/KV work and SLO feasibility	versioned workload trace and bins
fractional optimum rounded once	nearby architecture is equivalent	divisibility, memory, and token steps change cost/quality	enumerate discrete feasible candidates
median fit transformed once	median coefficients imply median decision	the nonlinear root solve does not commute with a coefficient-wise median, and the gate needs the upper tail of the requirements, not their center	solve every grouped-bootstrap draw, then train the $k_\delta$-th smallest requirement
per-draw horizon called a chance constraint	quality-feasibility probability looks high	each draw received its own horizon, so the probability counts draws where the target is reachable, not draws met by the one trained run	train the $k_\delta$-th smallest requirement and evaluate every draw at it
laws selected and certified on one hidden set	certified surface looks accurate	the certificate repeats the lucky scores that kept the laws	disjoint fit, selection, and final-test groups; a final miss aborts
quality equality assumed after quantization	smaller artifact is equal quality	artifact changed the evaluated model	consume M8 artifact-level quality gate
parameter count predicts decode cost	smaller always means proportionally cheaper	K/V traffic, batching, vocabulary head, and kernels matter	consume M5 measured runtime profile
source MFU copied	paper’s 50%/1% values describe current stack	hardware/software/workload changed	benchmark exact profile; label source numbers
high-demand recommendation extrapolates extreme ratios	more tokens always follow the fitted beta	fit support ended near ordinary ratios	include long-ratio evidence or target pilot
pilot tuned until pass	revised curve is now validated	validation evidence became development data	version refit and reserve new pilot
abort hidden	final successful run proves the process	selection bias removed failed forecasts	preserve forecast, miss, and trigger
M4.9 specifies scheduler/quantizer	lifecycle section is complete	ownership expanded into M5/M8	consume versioned outputs and cross-reference

0x02P Lifecycle Allocation Review Gate

Before authorizing a lifecycle allocation, answer:

• Is the comparison fixed-quality, or is a quality difference being sold as efficiency?
• Is $D_{req}(N,\ell)$ solved only where its residual allowance is positive?
• Are training-only and lifecycle-optimal objectives both shown?
• Are training, prefill, and decode separate cost terms?
• Are FLOPs, accelerator-seconds, currency, and energy kept as distinct units?
• Are hardware, precision, runtime, price, MFU/throughput, and workload versions frozen?
• Are source-reported 50%/50%/1% MFUs and 70/215 token lengths labeled as one paper scenario rather than defaults?
• Are prompt and output distributions represented rather than only total tokens?
• Is demand evaluated under low, base, high, and relevant stress scenarios?
• Is break-even demand calculated with one shared cost unit?
• Is the decision policy expected-value, robust-regret, tail-risk, or another preregistered objective?
• Are continuous optima mapped to complete architecture, token, data, hardware, and artifact configurations?
• Is one rounded horizon per architecture chosen from the per-draw requirements, and is every uncertainty draw evaluated at that same horizon?
• Are replica counts, memory admission, and capacity rounded inside every demand and runtime draw?
• Were the laws chosen on selection groups that are disjoint from the final hidden-scale groups that certify them?
• Are quality-surface, functional-form, demand, and runtime uncertainty all represented?
• Does every infeasible candidate retain a reason code?
• Does the chosen candidate report quality-feasibility probability, lifecycle cost intervals, winner probability, and scenario regret?
• Is the target-adjacent pilot selected to test the unstable decision boundary?
• Are pilot milestones and pass/abort/refit rules frozen before launch?
• Does a pilot used for refitting cease to count as final validation?
• Are the original prediction and failed gate preserved?
• Does M4.9 consume, rather than duplicate, M5 runtime and M8 artifact-release mechanics?

The lifecycle analysis closes the allocation workflow:

fit what quality costs to train
    -> price the exact deployment candidates under demand scenarios
    -> propagate fit, demand, and runtime uncertainty
    -> select from executable configurations
    -> freeze and test the decision near target scale
    -> authorize or reopen as a new study version

The terminal gate audits the complete article’s formulas, sources, responsive figures, cross-module boundaries, and final rendered page.

0xFC Final Source, Formula, Render, And Boundary Gate

The terminal audit does not add another scaling-law family. It tests whether the complete article preserves the contracts developed above when read, built, and rendered as one document.

Formula contract

• Are all display equations delimiter-balanced and rendered without a MathJax error?
• Is every symbol defined in the local derivation, the core symbol table, or an immediately adjacent contract table?
• Do the wide decision equations retain renderer-safe text fallbacks where a failed math renderer would otherwise hide the operational rule?
• Are analytic identities kept separate from empirical forms, fitted coefficients, approximations, and source-reported examples?
• Do all optimization results state their objective, constraints, positivity domain, and accounting convention?

Source contract

• Does every retained licensed source figure have a nearby source, figure number, license, and experimental-scope warning?
• Does every responsive reconstruction identify itself as a reconstruction rather than paper artwork or new empirical evidence?
• Are source-reported coefficients, run counts, model sizes, token counts, benchmark results, and cost examples still attached to their paper regime?
• Do the references point to original papers or official Transformer System owner articles rather than search pages or unsourced summaries?

Render contract

• Do every local image path, responsive source, and internal post link resolve?
• Does every image have meaningful alternative text?
• Are all original article SVGs valid XML and paired with a mobile layout?
• Do desktop and mobile renders contain the complete article, all equations, no broken images, and no page-level horizontal overflow?

Boundary contract

Owner	M4.9 consumes	M4.9 does not duplicate
M4.1	exact parameter, FLOP, byte, and tensor-shape ledgers	Transformer cost-model derivation
M4.7	fitted baseline laws, compute-optimal coordinates, numerical certificates, and grouped uncertainty draws	baseline scaling derivation or optimizer mechanics
M4.8	certified law set, frozen selection rule, resampling groups and strata, support limits, finite-data coordinates, and abort status; its own prediction draws as form-stability evidence (Section 0x02M)	extrapolation selection, repeated-data law, or dependence analysis
M5.1	measured prefill/decode profiles and K/V-sensitive serving behavior	scheduler, cache, and placement mechanisms
M6.1	stored-versus-active parameter and compute conventions	router, capacity, dispatch, and expert mechanics
M8.9	low-bit admission evidence: transformed artifact, loader/kernel ABI, and parity; consumed for candidate artifacts before the decision (Section 0x02F) and required again for the trained artifact after the training authorization	artifact construction, loader validation, kernel admission, and parity testing
M8.10	artifact qualification and release evidence	quantization, packaging, deployment release, and rollback workflow

A final run is authorized only after the empirical study, target-adjacent pilot, runtime profile, and artifact qualification all refer to compatible versions. Passing one owner’s gate cannot substitute for another owner’s evidence.

0xFD Figure Provenance And Rights

The article uses two different visual objects:

licensed source figure:
    attributed source evidence retained under an explicit reuse license

article-authored reconstruction:
    article-authored explanation of a mechanism or source-reported geometry;
    no paper artwork or digitized measurements; not new empirical evidence

An arXiv rights link records the terms selected for that submission. The arXiv non-exclusive distribution license is not described here as a general permissive reuse license, so source artwork governed only by that record is not served locally. CC-licensed figures retain the named license and attribution, and responsive adaptations are identified as adaptations. The Schaeffer source figure is linked rather than served under its noncommercial, no-derivatives license. The separate m4-7-metric-created-threshold visual is an independent article explanation of the metric transformation.

Source figures used	Desktop evidence	Mobile treatment	Rights record
Gadre et al., Figures 2, 3	attributed CC BY source figures	attributed responsive adaptations	CC BY 4.0
Schaeffer et al., Figure 2	source artwork linked, not locally served; independent article explanation used instead	no derivative served	CC BY-NC-ND 4.0
Sardana et al., Figure 1	article-authored semantic reconstruction; source artwork not served	independent responsive reconstruction	paper link retained; submission uses arXiv non-exclusive distribution

0xFF References

• M4.1 - Transformer Cost Model: Parameters, Activations, FLOPs, and Memory Traffic
• M5.1 - Autoregressive Serving State: KV Cache Lifecycle and Scheduler Pressure
• M6.1 - Routed Capacity: MoE Experts, Routers, and Token Dispatch
• M8.9 - Quantized Artifacts and Runtime Integration
• M8.10 - Deployment Evaluation, Release, and Rollback
• Hoffmann et al., “Training Compute-Optimal Large Language Models”, 2022.
• Gadre et al., “Language Models Scale Reliably with Over-Training and on Downstream Tasks”, 2024.
• Wei et al., “Emergent Abilities of Large Language Models”, 2022.
• Schaeffer et al., “Are Emergent Abilities of Large Language Models a Mirage?”, 2023.
• Sardana et al., “Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws”, 2024.