A statistical, systems, and release-engineering treatment of quantized deployment decisions: immutable candidate tuples, paired non-inferiority tests, workload-shaped latency and goodput curves, memory and energy ledgers, Pareto selection, shadow and canary release, production observability, and cache-safe rollback.
A derivation- and implementation-oriented treatment of LLM quantization outliers: geometric diagnostics, mixed-precision decomposition, SmoothQuant scaling, affine shifts, fixed and learned rotations, structured learned transforms, graph fusion, calibration, and runtime cost.
A derivation- and implementation-oriented treatment of scalar weight-only post-training quantization: round-to-nearest, layer reconstruction geometry, OBD/OBS/OBQ, GPTQ compensation and lazy block updates, AWQ activation-aware scaling, calibration state, packed artifacts, and runtime kernels.
A derivation- and implementation-oriented treatment of extreme weight-only compression: sensitivity-weighted nonuniform codebooks, dense-and-sparse exceptions, SpQR's hierarchical representation, AQLM additive codebooks, QuIP# incoherence and E8 lattice codes, VPTQ second-order vector quantization, artifact bit ledgers, and runtime decoder contracts.
A derivation- and implementation-oriented treatment of request-dependent quantization: dynamic activation scales, integer GEMM epilogues, attention scale placement, persistent KV-cache rate and metadata, streaming append state, KIVI, KVQuant, PrefixQuant, QServe W4A8KV4, production FP8 cache formats, and serving correctness.
A derivation- and implementation-oriented treatment of quantization-aware training: fake-quant arithmetic, straight-through gradients, learned step sizes and clipping, observer schedules, LLM-QAT data-free distillation, weight/activation/KV insertion sites, efficient QAT variants, distributed training state, and export parity.
A derivation- and implementation-oriented treatment of native low-bit Transformer design: BitLinear, binary and ternary weight codes, latent-weight optimization, variance control, BitNet b1.58, OneBit SVID and distillation, activation-lowering variants, distributed scale ownership, and the boundary between a code alphabet and an executable runtime.
A byte-, tensor-, graph-, and kernel-level treatment of quantized deployment artifacts: safetensors, GGUF, ONNX QDQ/QOperator, packing and alignment, sharding, converter and loader state machines, topology-aware placement, kernel admission, parity ladders, startup memory, security, and runtime integration across vLLM, TorchAO, TensorRT-LLM, and llama.cpp.
A mechanism-level comparison of major LoRA variants: quantized-base training, adaptive rank allocation, rank-stable scaling, factor-specific optimization, frozen-factor activation savings, SVD initialization, quantization-aware initialization, and magnitude/direction decomposition.
A derivation- and implementation-oriented foundation for model quantization: affine maps, error, scale shapes, quantized matrix multiplication, accumulator bounds, packed storage, kernel execution, and effective memory cost.
A derivation- and implementation-oriented treatment of quantization calibration: observer state, clipping objectives, representative corpora, output-aware error, sensitivity scans, Hessian alternatives, mixed-precision optimization, and runtime-feasible precision maps.