Step 5 of 15Method

Measure decode fairly

With a warm-up request first, steady decode was within 2% in every arm. The graph-less arms had moved 40 s to their first request, so wake to first answer improved only 7%.

Wake to first answer
~135 → ~125 s
Steady decode
within 2%
First request, no graphs
~44 s
Step 05 · Measured with the first answer
All graphs
135 s
Piecewise only
134 s
Piecewise, fewer sizes
124 s
No graphs
126 s

Hover or tap a segment for its time. The twin's readiness probe runs every 10 s, so totals land on 10-second steps and the last segment absorbs the wait.

Add the first answer and the graph-less arms' lead nearly disappears: their 44-second first request is the time they skipped at startup.

Problem

The decode benchmark timed the first requests after start, so it measured warm-up along with decoding.

Change

One timed warm-up request before the decode benchmark, reported on its own. Two runs per arm.

Result

Steady decode was within 2% in every arm. The graph-less arms paid about 40 seconds on their first request instead of at startup. For a caller, wake to first full answer went from about 135 seconds to about 125 at best: 7%, not 38%.

The same arms, judged three ways · Time to Ready

All graphs
130 s
Piecewise only
90–91 s
Piecewise, fewer sizes
80 s
No graphs
81 s

Switch views: a 38% shorter start shrinks to a 7% shorter wait once the first answer is on the clock.

The same arms, three ways. Steady decode is within 2% everywhere.

Decision

From here on, a change was judged on wake to first answer, not on time to Ready.