A source-audited study of extrapolation: scale-ordered validation, alternative scaling-law families, predictive model selection, repeated data, dependence, and effective-data constraints.
A source-audited study of over-training, downstream metric behavior, lifecycle cost, measured serving demand, uncertainty-aware artifact selection, pilot gates, and a training authorization that hands release and rollback requirements to the serving and deployment modules.
A source-audited study of neural scaling experiments: measurement contracts, power laws, joint model-data loss surfaces, compute-optimal allocation, paper-contract comparison, fitting, and uncertainty.
A tensor- and state-machine study of multimodal placeholder expansion, sequence packing, attention and loss masks, training stages, prefill/decode execution, media caching, batching, and safe reuse.
A derivation- and implementation-oriented treatment of quantization-aware training: fake-quant arithmetic, straight-through gradients, learned step sizes and clipping, observer schedules, LLM-QAT data-free distillation, weight/activation/KV insertion sites, efficient QAT variants, distributed training state, and export parity.
A derivation- and implementation-oriented study of Transformer MLPs: position-wise channel mixing, activation and GLU variants, SwiGLU width matching, backward flow, fused execution, knowledge interpretations, and memory alternatives.
A derivation- and implementation-oriented study of LoRA: low-rank update geometry, tensor shapes, forward and backward passes, initialization, scaling, optimizer state, target-module selection, and the limits of the low-rank assumption.
A derivation- and implementation-oriented study of multi-token prediction objectives, shifted labels, shared-trunk gradients, prediction-head designs, training memory, and the boundary between denser supervision and faster inference.
A derivation- and code-path reconstruction of Transformer training: shifted supervision, causal/padding/packed/loss masks, cross-entropy and smoothing, accumulation, Adam, Noam scheduling, mixed precision, distributed parallelism, checkpoints, and reproducibility.