A rigorous study of why low-rank adaptation can work and where it fails: intrinsic-dimension measurement, matrix manifolds, SVD reconstruction, factor redundancy, initialization gradients, NTK preservation, expressive-power bounds, feature learning, and forgetting.
A derivation- and implementation-oriented study of LoRA: low-rank update geometry, tensor shapes, forward and backward passes, initialization, scaling, optimizer state, target-module selection, and the limits of the low-rank assumption.
A mechanism-level comparison of major LoRA variants: quantized-base training, adaptive rank allocation, rank-stable scaling, factor-specific optimization, frozen-factor activation savings, SVD initialization, quantization-aware initialization, and magnitude/direction decomposition.