+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: Off Topic (https://www.datasculpting.ai/inference/forumdisplay.php?fid=27)
+--- Thread: desk setup thread on 5090 (/showthread.php?tid=881)
desk setup thread on 5090 - macropilot - 10-08-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of SGLang and Threadripper box felt surprisingly usable once I tuned around HBM constraints.
Feel free to tear it apart if you see something dumb.
Re: desk setup thread on 5090 - rocmmixer - 10-09-2026
Quote:Thought I'd post a small recipe because a lot of threads ask similar things.
Good point. That matches what I saw on M3 Max. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: desk setup thread on 5090 - thetatinker77 - 10-10-2026
A couple things I'd check first. If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: desk setup thread on 5090 - turbogrid11 - 10-10-2026
That matches what I was seeing. That matches what I saw on 7900 XTX.
Re: desk setup thread on 5090 - sparrowgrid - 10-10-2026
A couple questions before I guess too hard. Are you testing this on 5090 or something else?
Re: desk setup thread on 5090 - kernelpeak - 10-13-2026
that fixed it for me too.
Re: desk setup thread on 5090 - batchyard - 10-14-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tool calling. Not glamorous advice, but repeatable baselines save a lot of time.
Re: desk setup thread on 5090 - bytehall - 10-16-2026
same here on my side.
Re: desk setup thread on 5090 - fusedriver - 10-18-2026
same here on my side.
Re: desk setup thread on 5090 - cudascope - 10-18-2026
I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.