+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: General (https://www.datasculpting.ai/inference/forumdisplay.php?fid=25)
+--- Thread: new here, what's your stack on 5090 (/showthread.php?tid=1496)
Pages:
1
2
Re: new here, what's your stack on 5090 - quiettoken - 10-18-2026
I'd start with the simpler stack.
Re: new here, what's your stack on 5090 - routeline23 - 10-18-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: new here, what's your stack on 5090 - thetagrid15 - 10-18-2026
Maybe, but I had the opposite result. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Worth testing the simpler explanation before adopting the more interesting one.
Re: new here, what's your stack on 5090 - marbletrack - 10-18-2026
Good point. I ended up in about the same place using vLLM. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: new here, what's your stack on 5090 - quiettoken - 10-19-2026
Quote:Good point. I ended up in about the same place using vLLM. Worth rechecking your defaults though, because one bad assumption can make the rest of the
I don't think that's the whole story. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Worth testing the simpler explanation before adopting the more interesting one.
Re: new here, what's your stack on 5090 - rocmplane - 10-19-2026
Quote:[quote]Good point. I ended up in about the same place using vLLM. Worth rechecking your defaults though, because one bad assumption can make the rest
I hit almost the same issue last night.
Re: new here, what's your stack on 5090 - cacheline - 10-19-2026
Interesting, I hadn't tried that. That matches what I saw on 5090. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: new here, what's your stack on 5090 - turboarch95 - 10-19-2026
That matches what I was seeing. That matches what I saw on 7900 XTX. The biggest change for me was simplifying the stack instead of adding one more optimization layer. The logs were somehow calmer than I was.
Re: new here, what's your stack on 5090 - rocmrunner40 - 10-19-2026
Thanks, that helps. That matches what I saw on A6000. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: new here, what's your stack on 5090 - vectorport - 10-19-2026
I don't think that's the whole story. I've seen people blame batch size when the real issue was much more mundane. Could be a workload difference, but I wouldn't generalize too fast from one setup.