|

M6.3 - LoRA Foundations: Low-Rank Updates, Gradients, and Target Modules

A derivation- and implementation-oriented study of LoRA: low-rank update geometry, tensor shapes, forward and backward passes, initialization, scaling, optimizer state, target-module selection, and the limits of the low-rank assumption.