+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: Off Topic (https://www.datasculpting.ai/inference/forumdisplay.php?fid=27)
+--- Thread: desk setup thread (/showthread.php?tid=862)
desk setup thread - turbogrid11 - 10-07-2026
Did a bit of testing and thought I'd post the result here.
After a few runs on 5090, I ended up with decent performance, but the setup only really made sense once I adjusted around NVFP4.
Nothing scientific here, just practical notes from a real setup.
Re: desk setup thread - latentcraft - 10-07-2026
Tried to reproduce it just now. On 5090 with SGLang, I saw basically the same behavior. I expected the model choice to matter more than it did.
Re: desk setup thread - bytelab - 10-07-2026
I'd approach it in a boring order. On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: desk setup thread - thetaember38 - 10-08-2026
A couple things I'd check first. If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: desk setup thread - modrunner27 - 10-08-2026
worth checking the driver first.
Re: desk setup thread - vramstream - 10-09-2026
Quote:worth checking the driver first.
Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?