+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: General (https://www.datasculpting.ai/inference/forumdisplay.php?fid=25)
+--- Thread: small win thread on A6000 (/showthread.php?tid=671)
Pages:
1
2
Re: small win thread on A6000 - marbletrack - 10-06-2026
Quote:That matches what I was seeing. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at wh
What worked for me was pretty simple. If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: small win thread on A6000 - cozysmith - 10-06-2026
worth checking the driver first.
Re: small win thread on A6000 - plaingarden18 - 10-06-2026
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
Re: small win thread on A6000 - marbletrack - 10-06-2026
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.
If you post your exact settings, people here can probably narrow it down pretty fast.