A rigorous study of why low-rank adaptation can work and where it fails: intrinsic-dimension measurement, matrix manifolds, SVD reconstruction, factor redundancy, initialization gradients, NTK preservation, expressive-power bounds, feature learning, and forgetting.
A mechanism-level study of mixtures of LoRA adapters: routing granularity, joint versus post-hoc training, HydraLoRA factor sharing, router collapse, static and dynamic composition, continual low-rank state, and multi-adapter runtime integration.
A derivation- and implementation-oriented study of LoRA: low-rank update geometry, tensor shapes, forward and backward passes, initialization, scaling, optimizer state, target-module selection, and the limits of the low-rank assumption.
A mechanism-level comparison of major LoRA variants: quantized-base training, adaptive rank allocation, rank-stable scaling, factor-specific optimization, frozen-factor activation savings, SVD initialization, quantization-aware initialization, and magnitude/direction decomposition.
A systems study of serving many LoRA adapters over one base model: heterogeneous batching, segmented kernels, paging, residency, merge and hotswap semantics, request/KV compatibility, scheduling, observability, and failure recovery.