disaggregated inference thoughts: what are you seeing?

[vramport]

I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality. Turns out 'just one more test' was a lie.