X1.1 - Unitization and Geometry Metadata Across Modalities
A derivation- and implementation-oriented study of how images, documents, audio, video, tools, and actions become addressable units with aligned spatial, temporal, and structural metadata.
Tag: #multimodal
X1.2 - Multimodal Encoders, Projectors, Resamplers, and Fusion
A derivation- and implementation-oriented comparison of modality encoders, linear and nonlinear projectors, learned-query resamplers, prefix fusion, gated cross-attention, and their training and runtime contracts.
X1.3 - Multimodal Masks, Training, Runtime, and State
A tensor- and state-machine study of multimodal placeholder expansion, sequence packing, attention and loss masks, training stages, prefill/decode execution, media caching, batching, and safe reuse.
M6.8 - Conditional Compute Beyond FFN Experts: Attention, Depth, and Modalities
A tensor-level study of routing outside FFN experts: Mixture of Attention Heads, SwitchHead, Mixture-of-Depths, causal route prediction, per-layer KV consequences, MoD+MoE composition, and staged multimodal sparse training.
M0.0 - Roadmap
A reading guide to Transformer System: what each module explains, the questions behind its articles, and where to find their main sources.