Hover or tap a segment for its time. The twin's readiness probe runs every 10 s, so totals land on 10-second steps and the last segment absorbs the wait.
Problem
Capturing CUDA graphs still took 52 seconds of a 130-second start.
Change
By default vLLM captures piecewise graphs, which leave attention outside the
graph, and full graphs of the whole decode step. Four arms: fewer capture sizes,
piecewise graphs only, piecewise with fewer sizes, and no graphs at all with
--enforce-eager, which in this vLLM also turns off torch.compile.
Result
Fewer capture sizes alone barely moved the start, 120–130 seconds. Piecewise graphs only started in 90 seconds, piecewise with fewer sizes in 80, and no graphs in 81: 30–38% shorter than the 130-second reference, which already had the pinned KV cache. Batch-1 decode in those three arms appeared to fall from 52 to 14 tokens per second. With fewer sizes alone it held at 51.7.
Fewer CUDA graphs, as first measured on the twin · Time to Ready
Decision
Batch-4 decode was unchanged in every arm, about 51 tokens per second per sequence, and the benchmark ran batch 1 first. A 73% loss at batch 1 alone looked like first-use warm-up landing on the first measurement, so it was treated as a measurement error until shown otherwise.