M8.1 - Quantized Numbers: Scales, Zero Points, Error, and the Kernel Contract

M8.1 - Quantized Numbers: Scales, Zero Points, Error, and the Kernel Contract

0x00 From Model Growth To Quantized Arithmetic

Quantization is often introduced as a cast from a large dtype to a smaller one. That description starts too late. The real problem begins with the way a Transformer turns model scale into physical state.

A model with $P$ parameters stored at $b$ bits per parameter needs, for the raw parameter payload alone,

For a 70-billion-parameter model, that idealized payload is about 280 GB at FP32, 140 GB at FP16/BF16, 70 GB at eight bits, and 35 GB at four bits. Those numbers exclude tensor headers, scales, zero-points, padding, duplicated loading buffers, runtime workspaces, activations, and K/V cache. They nevertheless explain why numerical representation became a deployment question: changing the number of bits changes which devices can hold the model, how many replicas fit in a server, how much weight traffic each decode step must read, and how much capacity remains for request state.

The pressure is not only checkpoint size. The original A34 article correctly begins with a wider resource picture:

Quantization therefore belongs to model compression, but it is not synonymous with compression. Pruning removes or regularizes structure. Distillation trains a smaller student. Low-rank factorization changes a matrix representation. Architectural compression changes the network itself. Quantization keeps a tensor or operator interface approximately intact while replacing a large numerical state space with a smaller representable set. Later M8 articles will deliberately blur that boundary: sparse exceptions combine pruning-like structure with quantization, and native binary Transformers make the low-bit operator part of the architecture.

0x00A Before Neural Networks: Continuous Values, Finite Codes

The word quantization predates neural networks. In signal processing and numerical computing, it means mapping a continuous value, or a value drawn from a very large discrete set, to one member of a finite code set. Every stored computer number already performs some form of finite representation; model quantization changes the set and the mapping under a task-specific error budget.

Three numerical ideas must be separated before discussing INT8 or INT4:

1. range asks which magnitudes can be represented;
2. resolution asks how far apart adjacent representable values are;
3. code semantics ask how a stored bit pattern reconstructs a number.

Floating-point formats spend bits on sign, exponent, and significand. Their spacing changes with magnitude, which gives them broad dynamic range. Fixed-point and affine integer formats place uniformly spaced levels inside a selected interval; they trade global dynamic range for regular arithmetic and dense use of codes within that interval. A learned codebook can place nonuniform levels where a distribution or objective needs them, but then the decoder is no longer ordinary affine arithmetic.

This is why “four bit” is not a complete description. A signed INT4 grid, an FP4 format, a normal-float codebook, a ternary alphabet packed into two physical bits, and an index into a vector codebook all have different geometries. The storage width tells us how many code patterns are available. It does not tell us the represented values, the metadata, the accumulator, or the executable operator.

0x00B Why Neural Networks Can Tolerate Approximate Numbers

Quantization works at all because a trained network usually does not require every stored scalar to retain all source-format bits. Redundancy, distributed representations, normalization, and overparameterization can make many local perturbations tolerable. But that observation is statistical, not a guarantee. Different layers, channels, tokens, and operators can have very different sensitivity; errors can cancel, remain local, or compound through the residual stream.

The useful question is therefore not “does the quantized tensor resemble the source tensor?” It is a ladder:

scalar reconstruction:
    how far is each represented value from its source value?

operator reconstruction:
    how far is X * W_hat from X * W for relevant X?

block and model behavior:
    how do perturbations pass through normalization, attention,
    residual addition, nonlinearities, and autoregressive feedback?

runtime behavior:
    does the served kernel execute the intended representation,
    and does the resulting system improve the target workload?

The integer-arithmetic inference work of Jacob et al. made this co-design explicit: a quantization scheme, a training or calibration procedure, and an integer execution path form one system. Large language models make the same principle harder because their scale, activation outliers, autoregressive state, and prefill/decode asymmetry expose failure modes that smaller feed-forward deployments could often ignore.

0x00C What Is Actually Quantized?

A Transformer contains several numerical populations with different lifetimes:

Quantization can be static or dynamic, symmetric or affine, per-tensor or fine-grained, post-training or training-aware, weight-only or joint weight/activation/cache. These are independent axes rather than one progression from “simple” to “advanced.” M8 will separate them so that a method name cannot hide which tensor changed, when its parameters were chosen, and which operator consumes it.

0x00D The Deployment Object

M4.1 counted parameters, FLOPs, HBM traffic, activations, and K/V state. M5 then showed that a served model is more than its checkpoint: requests create persistent cache state, schedulers create temporary state, and kernels decide which theoretical savings become runtime savings.

M8 starts where those ledgers meet a new numerical representation.

The owner question is:

What exact object replaces a floating-point tensor, how does that object approximate the original arithmetic, and what must a runtime do before its smaller bit width becomes a real deployment benefit?

The answer is not merely INT4 or INT8. A quantized tensor is a contract:

quantized tensor =
    integer or low-precision codes
  + scales
  + optional zero-points
  + scale granularity and axis
  + packed layout
  + accumulator and output rules
  + a kernel that implements those rules

Reducing one field from 16 bits to 4 bits does not settle the rest of the path. The codes may be packed two per byte. Scales may remain FP16 or FP32. Zero-points may use another integer type. Products may accumulate in INT32 or floating point. The output may return to BF16. Some kernels unpack into registers and multiply with floating activations; others execute native low-precision matrix instructions.

This article derives that foundation. It does not choose calibration data, reproduce GPTQ or AWQ, solve activation outliers, or decide whether to quantize K/V. Those mechanisms need the numerical contract built here and receive separate M8 articles.

0x01 Five Precision Roles, Not One Dtype

For one linear layer, use the row-major mathematical convention:

where:

A deployment record must distinguish at least five roles:

W4A16 identifies weight and activation widths, but still leaves open:

integer or codebook representation?
symmetric or affine?
per-tensor, per-channel, or per-group scales?
what group size and axis?
which scale and zero-point dtypes?
which accumulator?
which packed ordering?
which kernel and hardware path?
which tensors are excluded?

That is why two checkpoints both described as W4A16 need not have the same quality, size, load path, or latency.

0x01A Integer And Floating Low Precision Solve Different Problems

An integer code has uniformly spaced integer values. A floating format divides bits among sign, exponent, and significand, creating nonuniform spacing over a much larger dynamic range.

The words “4-bit” therefore do not identify one numeric system. INT4, FP4, and a 16-entry codebook all expose 16 codes, but those codes reconstruct different values and require different kernels.

0x01B Exponent Bits And Fraction Bits Exchange Range For Resolution

For a normalized binary floating-point value, the conceptual reconstruction is:

where $e$ is the stored exponent and $f$ is the stored fraction. More exponent bits cover more orders of magnitude. More fraction bits place more representable values between adjacent powers of two.

The exact special values and finite ranges depend on the concrete format variant, so a deployment artifact must name more than FP8.

For integers, the code itself has no exponent. The external scale supplies real-world range:

This separates the compact code from the range metadata. A group with one very large outlier increases $s$ for every ordinary value sharing that group; a floating format can instead use a larger exponent for the outlier while retaining magnitude-relative spacing. That difference is one reason integer and floating low-precision schemes react differently to outliers.

Subnormal handling, flush-to-zero behavior, NaN/Inf policy, and accumulator precision can also change effective arithmetic. Those are execution-contract fields, not footnotes.

0x02 Deriving Uniform Affine Quantization

Suppose a real interval $[r_{min}, r_{max}]$ must be represented by integer codes in $[q_{min}, q_{max}]$.

We want a linear reconstruction:

where:

s > 0:
    real units represented by one integer step

z:
    integer code that reconstructs real zero

The endpoint requirements are:

Subtract the first equation from the second:

so the scale is:

Substitute back into the lower endpoint equation:

The hardware zero-point must be an integer code, so the stored value is usually:

Quantization first converts real units into step units, rounds to the integer lattice, translates by $z$, and saturates to the code range:

Dequantization reverses the translation and restores real units:

Renderer-safe contract:

s = (r_max - r_min) / (q_max - q_min)
z = clamp(round(q_min - r_min / s), q_min, q_max)
q = clamp(round(x / s) + z, q_min, q_max)
x_hat = s * (q - z)

0x02A Why The Zero-Point Matters

If $z$ is representable, then:

Exact zero matters for padding, zero-valued activations, and operations whose algebra assumes zero remains zero. The rounded $z$ can shift the representable real endpoints slightly; the interval and code range cannot always satisfy both exact endpoints and exact zero simultaneously after integer rounding.

The affine map is useful when the real range is not centered around zero. For example, ReLU-like nonnegative activations can use most of an unsigned code range rather than spending half of a signed range on negative values that never occur.

0x02B Symmetric Quantization Is A Special Case

For a signed $b$-bit symmetric quantizer, choose:

and use codes $[-Q,Q]$. The most negative two’s-complement code is commonly left unused so positive and negative magnitudes have equal reach.

Let:

Then:

and:

Symmetry removes zero-point correction terms from matrix multiplication. The tradeoff is range efficiency: a strongly shifted tensor may waste codes on an unused side of zero.

0x02C A Complete Four-Bit Trace

Quantize:

x = [-1.00, -0.60, 0.00, 0.55, 1.00, 1.40]
real clipping range = [-1.00, 1.40]
unsigned 4-bit code range = [0, 15]

The scale is:

The ideal zero-point is:

so store $z=6$. The trace is:

The upper reconstructed endpoint is $1.44$, not exactly $1.40$, because the integer zero-point was rounded. This is not a bug in the table. It is part of the affine contract.

0x02D Rounding Is A Policy

round is not one universal operation.

For the step-unit value x/s exactly halfway between two integers, common policies include:

For stochastic rounding, let $u=x/s$ and $a=\lfloor u\rfloor$. A basic rule is:

Then:

before clipping. This can matter during low-precision training, where small updates can repeatedly fall below one deterministic quantization step. Deterministic inference formats usually favor reproducible nearest rounding, but the exact tie rule must still match converter and kernel.

0x02E Constant And Nonfinite Groups Need Explicit Handling

If $r_{max}=r_{min}$, the ordinary scale formula divides by zero. A converter needs a defined constant-group policy, for example:

if every value is zero:
    store all zero codes and a conventional nonzero scale

if every value is the same nonzero constant:
    choose a valid scale/zero pair that reconstructs the constant,
    or mark the tensor/group as an exception

Likewise, NaN and infinity cannot silently enter min/max reductions. A serious exporter either rejects the tensor, sanitizes values under a documented policy, or stores an exceptional representation. Allowing one nonfinite value to determine a scale can corrupt an entire group.

0x03 Error Has Two Mechanisms

Define the clipped real value:

The total quantization error is:

Insert $x_c$:

The first term is rounding error. The second is clipping error.

Inside the representable interval, nearest rounding gives:

except for small endpoint effects caused by zero-point rounding. Outside the interval, clipping error can be arbitrarily larger than $s/2$.

This creates the central range tradeoff:

wider clipping range:
    fewer clipped outliers
    larger step s
    larger rounding error for ordinary values

narrower clipping range:
    smaller step s
    better resolution in the center
    larger clipping error in the tails

0x03A Where The s^2 / 12 Rule Comes From

A common approximation is:

It follows only under a specific model. Assume no clipping and assume rounding error is uniformly distributed:

Its mean is zero. Its second moment is:

Evaluating the integral:

The rule is therefore an approximation, not a universal quantization law. It can fail when:

values concentrate near grid boundaries
error correlates with the signal
the tensor has only a few values
clipping is significant
rounding is stochastic or biased
the codebook is nonuniform
downstream sensitivity weights some coordinates more than others

0x03B Tensor Error Is Not Output Error

For the linear layer:

the output perturbation is:

Two weight tensors with the same $\|E_W\|_F$ can produce different output errors because the activation directions in $X$ weight the residual differently. This observation is the bridge to calibration-aware and Hessian-aware methods in M8.2 and M8.4.

It also explains why minimizing local weight MSE is a baseline objective, not a complete model-quality objective.

0x04 Granularity Is A Tensor-Shape Contract

The quantizer is incomplete until it says which values share $(s,z)$.

Use:

X: [M, K]
W: [N, K]

Common contracts are:

This article defines:

G = number of weight values in one group along K
N_g = ceil(K / G) groups per output row
N_groups = N * ceil(K / G) groups in this [N, K] weight tensor

Groups restart at each output row; a short final group still stores its own metadata. They do not span row or tensor boundaries.

Therefore:

G = K:
    one group per output row
    equivalent to per-output-channel scaling

G = 1:
    one scale per individual weight
    finest possible scalar granularity
    usually an impractical metadata cost

This definition avoids an ambiguity in which “group count” and “group size” are accidentally inverted.

0x04A Groupwise Tensor Trace

Let:

M = 2 token rows
K = 8 input features
N = 3 output rows
G = 4 values per weight group
N_g = 2 groups per output row

Then:

X:       [2, 8]
Q_W:     [3, 8]
s_W:     [3, 2]
z_W:     [3, 2]
Y:       [2, 3]

The metadata lookup for weight $W_{n,k}$ is:

Pseudocode:

def dequant_weight(qw, scales, zeros, group_size):
    n_rows, K = qw.shape
    out = empty_float_like(qw)

    for n in range(n_rows):
        for k in range(K):
            g = k // group_size
            out[n, k] = scales[n, g] * (qw[n, k] - zeros[n, g])

    return out

Real packed formats may permute rows, interleave groups, pad $K$, or fuse zero-point encodings. The logical shapes still need to be recoverable from metadata.

0x04B Finer Is Not Automatically Better

Reducing $G$ lets each scale fit a smaller local range, often reducing reconstruction error. It also creates:

more scale and zero-point bytes
more metadata loads
more scale changes inside a reduction
more complex packing constraints
possible tensor-core or vectorization misalignment
larger conversion artifacts

Granularity is therefore a quality, storage, and kernel decision at the same time.

0x05 Deriving Quantized Matrix Multiplication

Assume one activation scale and zero-point per token row and one weight scale and zero-point per output row:

Start from one output element:

Substitute both reconstructions:

Expand the product:

This derivation explains why zero-points are not free metadata. An asymmetric integer GEMM needs correction terms, precomputed sums, fused epilogues, or an equivalent centered representation.

For symmetric quantization:

so:

The integer matrix product produces an accumulator matrix:

A_int: [M, N]

The activation scales have shape [M, 1], the weight scales have shape [1, N], and their outer product rescales the accumulator:

0x05A Groupwise Scales Change The Reduction

If weight scale changes every $G$ values, it cannot be moved outside the entire $K$ reduction.

For symmetric groupwise weights ($z^W_{n,g}=0$), retain the activation row’s zero-point unless activations are also symmetric:

Here $\mathcal{K}_g$ contains the valid indices in that row’s group, including a possibly short tail. Only when $z^X_m=0$ may the activation subtraction disappear. For example, activation codes [3, 4] with zero-point 2, weight codes [1, 1] with zero-point 0, and unit scales give (3-2)*1 + (4-2)*1 = 3, not 7.

The kernel must form a partial accumulator for each group, apply that group’s scale, and then combine group contributions. Smaller groups can improve approximation while increasing scale traffic and epilogue work.

0x05B Accumulator Width Is Part Of Correctness

For symmetric signed INT8 codes in $[-127,127]$, one product has maximum magnitude:

A conservative worst-case sum bound is:

The maximum $K$ that fits a signed INT32 accumulator under that worst-case alignment is:

This does not mean every INT8 dot product near that length overflows. Real signs cancel and kernels accumulate tiles. It means accumulator safety cannot be inferred from operand width alone.

For affine UINT8 values, centered magnitudes can reach 255 depending on $z$, producing a different bound. INT4 has a smaller product bound, but packing and instruction behavior differ. Bias addition, partial-reduction combination, and output conversion must also declare their dtypes.

0x05C Numeric GEMM Trace

Use one activation row and two output rows:

M = 1, K = 4, N = 2

q_X = [ 2, -3,  4, 1]
s_X = 0.10
z_X = 0

q_W[0] = [ 1,  2, -1, 3]
s_W[0] = 0.20

q_W[1] = [-2,  1,  2, 0]
s_W[1] = 0.25
z_W = 0

The integer dot products are:

The scale outer product is:

s_X * s_W = [0.10 * 0.20, 0.10 * 0.25]
            = [0.02, 0.025]

Therefore:

The reconstructed floating operands give the same result:

X_hat    = [ 0.2, -0.3,  0.4, 0.1]
W_hat[0] = [ 0.2,  0.4, -0.2, 0.6]
W_hat[1] = [-0.5, 0.25,  0.5, 0.0]

X_hat @ W_hat.T = [-0.10, 0.025]

This equality is between the integer path and the reconstructed quantized operands. It does not say the result equals the original unquantized layer output. That remaining difference is quantization error.

0x06 The Packed Kernel Path

A checkpoint does not normally store a dense integer tensor with one byte per INT4 value. It packs codes and arranges them for a target loader or kernel.

For two unsigned 4-bit codes $a,b\in[0,15]$:

Unpack with:

Signed values may use two’s-complement nibbles, an offset encoding, or another format-specific convention. Nibble order is part of the artifact ABI.

0x06A Offline Conversion

def quantize_and_pack_weight(W, bits, group_size, symmetric):
    N, K = W.shape
    qmin, qmax = code_range(bits, symmetric)
    groups = ceil_div(K, group_size)

    codes = empty_int_matrix(N, K)
    scales = empty_float_matrix(N, groups)
    zeros = empty_integer_matrix(N, groups) if not symmetric else None

    for n in range(N):
        for g in range(groups):
            lo = g * group_size
            hi = min(K, lo + group_size)
            block = W[n, lo:hi]

            scale, zero = choose_affine_parameters(
                block,
                qmin=qmin,
                qmax=qmax,
                symmetric=symmetric,
            )
            q = clamp(round(block / scale) + zero, qmin, qmax)

            codes[n, lo:hi] = q
            scales[n, g] = scale
            if zeros is not None:
                zeros[n, g] = zero

    packed, layout = pack_for_target_kernel(codes, bits)
    return QuantizedTensor(
        packed=packed,
        scales=scales,
        zeros=zeros,
        logical_shape=(N, K),
        group_size=group_size,
        layout=layout,
    )

choose_affine_parameters is deliberately a replaceable policy. M8.2 owns how clipping ranges and mixed precision are selected. M8.4 owns methods that reconstruct weights using activation or curvature information rather than plain local range mapping.

0x06B Three Runtime Execution Families

1. Full dequantization before the operator

packed checkpoint
    -> allocate full fp16/bf16 W_hat
    -> run ordinary GEMM

This can reduce disk size but loses most weight-residency and HBM-traffic savings during execution.

2. Fused unpack/dequantize inside the operator

load packed weight tile
    -> unpack in registers/shared memory
    -> apply group metadata
    -> multiply with fp16/bf16 activations
    -> accumulate

A revision-scoped example is TensorRT-LLM v1.2.1, revision 376f7e1bd8ed, specifically its TensorRT engine/plugin path. The retained precision guide describes INT4/INT8 weights reconstructed during linear operations while activations remain FP16 or BF16. The corresponding WeightOnlyQuantMatmul plugin implements FP16 paths and build-conditional BF16 paths.

This example names the plugin-enabled path, not every execution of that release: its graph helper also contains a separate dequantize-plus-matmul branch when the plugin is disabled or transposition is requested. This is source-verified behavior, not a local GPU dispatch or performance measurement, and it is not a claim about the PyTorch backend or every later release. M8.9 must still record the actual backend and selected kernel for a deployment.

3. Native low-precision multiply-accumulate

quantized activation tile + quantized weight tile
    -> low-precision matrix instruction
    -> wider accumulator
    -> scale/correction epilogue

This path can reduce both operand traffic and arithmetic width, but only for supported datatype, shape, layout, and hardware combinations.

Teaching kernel sketch:

def quantized_linear_tile(X_tile, QW_packed, meta, bias):
    # Exact order and tensor-core fragment layout are backend-specific.
    qweight = unpack_fragment(QW_packed, meta.layout)

    if meta.weight_only:
        weight = dequant_fragment(qweight, meta.scale, meta.zero)
        acc = mma_float_activation_weight(X_tile, weight)
    else:
        qx, sx, zx = quantize_activation_fragment(X_tile, meta.activation_rule)
        int_acc = mma_low_precision(qx, qweight)
        acc = apply_scale_and_zero_corrections(int_acc, sx, zx, meta)

    return cast_output(add_bias(acc, bias), meta.output_dtype)

The code is not a reproduction of a CUDA kernel. It identifies the state that a correct implementation must carry.

0x06C A Source-Reported Speedup Is Not A Datatype Constant

A34 cites a secondary discussion of a large acceleration result. The canonical source is Who Says Elephants Can’t Run, a production study of large sparse MoE inference. The paper reports up to 26x throughput improvement while quantizing expert weights to INT4 and applying a broader inference framework. That result is useful evidence that representation, placement, and system design can combine into a large gain. It is not evidence that FP32 -> INT4 makes an arbitrary dense GEMM or end-to-end service 26x faster.

Keep the claim decomposition explicit:

paper system:
    sparse MoE execution
    expert placement and inference optimizations
    INT4 expert-weight storage/decoding
    paper-specific model, batch, hardware, and baseline

claim that transfers:
    low-bit expert weights can remove an important memory barrier
    when the runtime co-design preserves the compact representation

claim that does not transfer automatically:
    26x for another model, operator, GPU, service, or baseline

This is why M8.1 distinguishes representation, kernel path, phase, and deployment tuple before assigning a speed number.

0x07 Effective Bits And Memory Ledgers

The ideal code payload for $P$ values at $b_q$ bits is:

For the row-bounded weight layout [N, K] defined above, let:

P = N * K
N_groups = N * ceil(K / G)
b_s = bits per stored scale
b_z = bits per stored zero-point

For multiple tensors, sum each tensor’s own group count; do not flatten across rows or tensor boundaries. The shortcut ceil(P / G) applies only to a different contract that groups a flat sequence without those boundaries. For N=3, K=5, and G=4, row-bounded grouping needs 3 * 2 = 6 metadata entries, not ceil(15 / 4) = 4.

Then a first metadata-aware estimate is:

Ignoring the last three terms, the exact logical code-plus-metadata bit rate for $P>0$ is:

This counts one stored scale and optional zero-point per group, with no secondary compression. For full groups (or as a large-tensor approximation when tail overhead is negligible), it reduces to:

Renderer-safe ledger:

logical_bits_per_value = code_bits + N_groups * (scale_bits + zero_bits) / P
effective_bits_per_value ~= code_bits + (scale_bits + zero_bits) / group_size

In the 3 x 5, G=4 example, INT4 codes plus six FP16 scales require 15*4 + 6*16 = 156 logical bits, or 10.4 bits/value, before packing. A tightly packed tensor-wide code stream needs ceil(60/8) = 8 bytes plus 12 scale bytes: 20 bytes total. If each row’s codes start on a byte boundary, codes instead need 3*ceil(20/8) = 9 bytes, for 21 total. These are two declared layouts, not interchangeable estimates; count byte rounding, padded groups and alignment once in the physical ledger.

0x07A A 70B Parameter Example

For 70 billion weights, symmetric INT4, $G=128$, and one FP16 scale per group, assume full groups or negligible row-tail overhead:

The payload is approximately:

not the metadata-free 35 GB.

Add an 8-bit zero-point per group:

which is about 36.64 GB before padding, headers, unquantized tensors, and runtime state.

This difference is small relative to 70B parameters, but it grows as $G$ shrinks. At $G=32$, FP16 scale plus UINT8 zero-point costs another $0.75$ bit per weight.

0x07B Checkpoint Memory Is Not Served Memory

A served process may hold:

M6.4 owns QLoRA and explains the training-state ledger: a quantized frozen base does not erase adapters, activations or optimizer state. Serving K/V is a separate runtime cost. M6.3 supplies the vanilla LoRA foundation, not the QLoRA mechanism.

0x08 Why A Smaller Tensor May Not Be Faster

M4’s roofline discipline still applies.

Quantization can improve runtime through two different channels:

bandwidth path:
    move fewer weight bytes from HBM

compute path:
    execute more low-precision products per cycle

The first can help decode-like GEMV and small-batch operations that repeatedly stream weights. The second requires supported low-precision instructions and enough work to use them efficiently.

New overheads include:

unpacking codes
loading scales and zero-points
computing dynamic activation scales
zero-point correction
converting accumulator/output dtype
padding and layout transforms
kernel dispatch
unsupported operators falling back to higher precision

A useful inequality is:

Quantization accelerates the operator only when this total is less than the original path.

0x08A Prefill And Decode Need Separate Measurements

For prefill, larger $M$ can make the matrix multiplication compute-intensive. A weight-only kernel that reconstructs values but does not use a faster matrix instruction may save bandwidth without improving the dominant compute path.

For one-token decode, $M$ is small and weight traffic is often prominent. Packed weight-only formats can be valuable, but launch overhead, group metadata, and batch shape still matter.

Therefore this claim is invalid:

4-bit checkpoint is 4x smaller than bf16
therefore end-to-end generation is 4x faster

The checkpoint ratio is not a latency model.

0x08B Backend Support Is Part Of The Method

Current serving engines expose multiple quantization families, and their hardware support differs. vLLM documents separate paths for GPTQ-style formats, AWQ, bitsandbytes, FP8, INT8, model-optimizer formats, quantized K/V, and others. TensorRT-LLM likewise documents recipe- and hardware-specific precision support: at v1.2.1 (revision 376f7e1bd8ed), its quantization guide lists FP8, FP4, AWQ and GPTQ recipes against separate model and hardware support matrices (Ampere through Blackwell), and its usage section names the default PyTorch backend. For TensorRT-LLM, the pinned engine/plugin example in 0x06B is not a support claim for a different backend or revision; check the selected recipe, hardware and dispatch. llama.cpp’s quantization tooling records per-tensor types and optional importance matrices in its artifact flow.

The exact support matrix changes with runtime version. The reproducible statement is:

runtime version + hardware + model + quantization format + selected kernel

not “the runtime supports 4-bit” in the abstract.

0x09 Design Choices That Share The Same Vocabulary

Several independent axes are often collapsed into one word.

These alternatives should not be ranked without a target workload.

0x09A Uniform Versus Nonuniform

Uniform affine quantization has a cheap reconstruction: subtract $z$, multiply by $s$. A nonuniform codebook can allocate more levels where values are dense, but reconstruction may require lookup or specialized instructions. NF4 in QLoRA is one example of a nonuniform 4-bit datatype designed around normalized weight distributions; it is not numerically equivalent to INT4.

0x09B Static Versus Dynamic

Static parameters are fixed before serving. Dynamic quantization computes some parameters from live tensors. Dynamic per-token activation scaling can follow request-dependent ranges, but computing reductions and scales becomes runtime work. M8.2 owns calibration and range-selection methods; M8.6 owns the live activation and K/V path.

0x09C PTQ Versus QAT

Post-training quantization converts a trained model. Quantization-aware training inserts simulated or real low-precision effects during optimization so parameters can adapt. That difference changes gradients, observers, export, and training cost. M8.7 owns those details.

0x0A A Minimal Reference Quantizer

This standard-library Python reference executes the affine map in 0x02, using a caller-declared range rather than estimating extrema from the input. It accepts a scalar or a nonempty list/tuple of built-in integers/floats; scalar input returns scalar codes/reconstruction, sequences return lists. Arithmetic uses Python floats and round (nearest, ties to even before adding the zero-point). It is a formula oracle, not a packed kernel or a bit-exact model of FP16 metadata.

The declared range must have positive finite width and produce a positive finite scale and finite decoded endpoints. Degenerate declared ranges, nonfinite values, empty inputs and unrepresentable arithmetic are rejected, not silently repaired. Constant input data is allowed with a valid declared range: for all zeros, the caller can supply [-1, 1]; a nonzero constant needs a deliberate range too. Range selection remains M8.2’s responsibility.

As in 0x02, saturation is to the code range after rounding, not a separate pre-clamp to rmin/rmax. Rounded/clamped zero-points can shift the decoded endpoints (especially for ranges excluding zero). The reference reports the chosen scale and zero-point so that shift remains visible. The signed option uses the full two’s-complement range; it does not implement the restricted symmetric [-Q, Q] policy of 0x02B.

import math


def finite_float(value, name):
    if type(value) not in (int, float):
        raise ValueError(name + " must be a built-in int or float")
    try:
        value = float(value)
    except OverflowError as error:
        raise ValueError(name + " is not representable as a float") from error
    if not math.isfinite(value):
        raise ValueError(name + " must be finite")
    return value


def affine_quantize(x, bits, rmin, rmax, signed=False):
    # This small oracle deliberately supports only 1..32-bit integer codes.
    if type(bits) is not int or not 1 <= bits <= 32:
        raise ValueError("bits must be an integer in [1, 32]")
    if type(signed) is not bool:
        raise ValueError("signed must be a bool")
    scalar = type(x) in (int, float)
    if not scalar and (type(x) not in (list, tuple) or not x):
        raise ValueError("x must be a scalar or a nonempty list/tuple")
    values = [finite_float(v, "x") for v in ([x] if scalar else x)]
    rmin = finite_float(rmin, "rmin")
    rmax = finite_float(rmax, "rmax")
    if signed:
        qmin = -(1 << (bits - 1))
        qmax = (1 << (bits - 1)) - 1
    else:
        qmin = 0
        qmax = (1 << bits) - 1

    width = rmax - rmin
    if not math.isfinite(width) or width <= 0:
        raise ValueError("quantization range must have positive finite width")

    scale = width / (qmax - qmin)
    if not math.isfinite(scale) or scale <= 0:
        raise ValueError("scale underflow or overflow")
    zero_real = qmin - rmin / scale
    if not math.isfinite(zero_real):
        raise ValueError("zero-point calculation is not finite")
    zero = min(qmax, max(qmin, round(zero_real)))
    lower = scale * (qmin - zero)
    upper = scale * (qmax - zero)
    if not math.isfinite(lower) or not math.isfinite(upper):
        raise ValueError("decoded endpoint overflow")

    def encode(value):
        # Saturate early at decoded endpoints to avoid value/scale overflow
        # for huge finite outliers; interior values still use the 0x02 map.
        if value <= lower:
            return qmin
        if value >= upper:
            return qmax
        return min(qmax, max(qmin, round(value / scale) + zero))

    codes = [encode(v) for v in values]
    reconstructed = [scale * (q - zero) for q in codes]

    return {
        "codes": codes[0] if scalar else codes,
        "scale": scale,
        "zero": zero,
        "reconstructed": reconstructed[0] if scalar else reconstructed,
    }

Assertions:

result = affine_quantize(
    x=[-1.0, -0.6, 0.0, 0.55, 1.0, 1.4],
    bits=4,
    rmin=-1.0,
    rmax=1.4,
    signed=False,
)

assert result["zero"] == 6
assert result["codes"] == [0, 2, 6, 9, 12, 15]
assert min(result["codes"]) >= 0
assert max(result["codes"]) <= 15
assert result["reconstructed"][2] == 0.0
assert result["scale"] * (result["zero"] - result["zero"]) == 0.0

# Unit scale and zero-point 1: ties are rounded before that translation.
ties = affine_quantize([-0.5, 0.5, 1.5], 2, -1.0, 2.0)
assert ties["codes"] == [1, 1, 3]
assert affine_quantize([0.0, 0.0], 4, -1.0, 1.0)["reconstructed"] == [0.0, 0.0]

Production quantizers must additionally bind these numerical policies to their actual dtypes and execution path, and define:

converter/kernel rounding parity
nonfinite and constant-group export policy
tail-group and padding policy
metadata-dtype scale underflow/overflow handling
parallel reduction determinism
packing order and endianness
tensor exclusions
metadata serialization

0x0B Reproducibility Record

A quantized artifact should record:

source model:
    checkpoint, revision, tokenizer, architecture config

target tensors:
    exact module names, exclusions, tied-weight treatment

numeric map:
    code family, bit width, signedness, q_min/q_max
    symmetric/asymmetric rule
    scale and zero-point dtypes
    clipping/range-selection rule
    rounding mode

granularity:
    scale axis
    values per group G
    tail policy
    scale and zero tensor shapes

artifact:
    packing order, layout version, padding, checksums
    per-tensor format exceptions

runtime:
    loader and framework version
    selected kernel/backend
    product, accumulator, and output dtypes
    prefill/decode batch shapes
    fallback operators

validation:
    reconstruction tests
    layer-output error
    end-task quality
    memory residency
    TTFT, TPOT, throughput, and energy

Without this record, INT4 group size 128 is not enough to reconstruct the experiment.

0x0C Failure Lab

0x0D Boundaries To The Rest Of M8

M8.1 owns the numerical and kernel contract.

Later articles own:

M8.2:
    calibration data, clipping objectives, observers,
    sensitivity, mixed precision, and precision maps

M8.3:
    activation outliers and equivalent transforms,
    including LLM.int8(), SmoothQuant, QuaRot, SpinQuant, and FlatQuant

M8.4:
    scalar weight-only PTQ reconstruction,
    including RTN, OBQ/OBS lineage, GPTQ, and AWQ

M8.5:
    extreme weight representations and very-low-bit methods,
    including sparse exceptions, codebooks, additive quantizers,
    incoherence processing, and vector quantization

M8.6:
    live activation and KV-cache quantization paths

M8.7:
    quantization-aware training and optimization through fake quantizers

M8.8:
    architectures trained natively with binary/ternary constraints

M8.9:
    artifact packaging, loader state, and kernel compatibility

M8.10:
    evaluation gates, promotion, rollback, and the deployment decision record

This expanded split is deliberate. A34-A36 contain too much mechanism diversity for one generic PTQ article without compressing the derivations, implementation traces, method alternatives, training/runtime bridge, and paper comparisons that Transformer System treats as non-compressible.

0x0E Claim Status

0x0F Publish Gate

0xFF References

• Transformer Series A34 - Quantization Fundamentals
• Transformer Series A35 - Large-Model Quantization Fundamentals
• Transformer Series A36 - Large-Model Quantization Schemes
• Compression Series 07 - Quantization As The First Serious Baseline
• M4.1 - Transformer Cost Model
• M6.4 - Low-Rank Adaptation Design Space
• Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
• A Survey of Quantization Methods for Efficient Neural Network Inference - canonical background source for quantization representations, PTQ/QAT tradeoffs, and hardware-aware inference. [SURVEY]
• Who Says Elephants Can’t Run: Bringing Large Scale MoE Models into Cloud Scale Production - source-reported MoE production framework, INT4 expert weights, and up-to-26x throughput result; not a universal datatype speedup. [PAPER]
• TensorRT-LLM v1.2.1 retained precision guide - revision-pinned TensorRT-path example, not an evergreen backend support matrix.
• TensorRT-LLM v1.2.1 WeightOnlyQuantMatmul plugin - implementation evidence for the scoped example; actual dispatch and performance require separate runtime checks.
• TensorRT-LLM v1.2.1 quantization guide - revision-pinned recipe list with model and hardware support matrices; its usage section addresses the default PyTorch backend, not the TensorRT engine/plugin path in 0x06B, and later revisions may differ.
• vLLM Quantization documentation
• llama.cpp quantization tooling