high-context local setup

[latentgarden]

I'd approach it in a boring order. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.

[mangopath4]

Can you fill in a bit more context? Is this local-only or are you routing anything externally?

[bytelab]

I'd approach it in a boring order. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.

[quietcraft]

Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.

[embershed99]

Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.

[deepseeked]

same here on my side.