M5.2 - Continuous Batching and Scheduler Pressure
A serving-scheduler study note on static batching failure, iteration-level scheduling, continuous batching, chunked prefill, TTFT, TPOT, goodput, and KV-aware admission control.
A serving-scheduler study note on static batching failure, iteration-level scheduling, continuous batching, chunked prefill, TTFT, TPOT, goodput, and KV-aware admission control.