small win thread on A6000

[marbletrack]

That matches what I was seeing. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at wh

What worked for me was pretty simple. If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[cozysmith]

worth checking the driver first.

[plaingarden18]

A couple questions before I guess too hard. Is this local-only or are you routing anything externally?

[marbletrack]

Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.

If you post your exact settings, people here can probably narrow it down pretty fast.