long context that stayed useful

[sanepeak48]

worth checking the driver first.

I think the bottleneck is somewhere else. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Worth testing the simpler explanation before adopting the more interesting one. At this point the real bottleneck is my patience.

[ggufvault]

What worked for me was pretty simple. On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.

[turboscope5]

My first guess would be this: If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[edgequill]

Did a small comparison this afternoon. On dual 3090 with LM Studio, I saw basically the same behavior. The main thing that moved was latency, not quality.

[torchkite]

that fixed it for me too.