M5.6 - Prefill/Decode Disaggregation and KV Placement
A systems-oriented study note on prefill/decode interference, chunked prefill, disaggregated serving, KV transfer, placement, routing, goodput, and exact cache handoff contracts.
A systems-oriented study note on prefill/decode interference, chunked prefill, disaggregated serving, KV transfer, placement, routing, goodput, and exact cache handoff contracts.