+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: Systems & Builds (https://www.datasculpting.ai/inference/forumdisplay.php?fid=29)
+--- Thread: Best workstation layout for local models (/showthread.php?tid=974)
Best workstation layout for local models - cozygrid12 - 10-09-2026
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once tokens/sec becomes important.
Trying to avoid rebuilding the whole thing twice.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: Best workstation layout for local models - homelabhenry - 10-09-2026
Quote:Spent some time on this today and wanted to compare notes.
I'd approach it in a boring order. On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Best workstation layout for local models - forgemixer - 10-09-2026
Ran a quick check on my box. On Threadripper box with llama.cpp, I saw basically the same behavior. The main thing that moved was latency, not quality.
Re: Best workstation layout for local models - phasenode - 10-09-2026
Appreciate the detail. I ended up in about the same place using LM Studio.
Re: Best workstation layout for local models - bytegarden73 - 10-12-2026
Did a small comparison this afternoon. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
Re: Best workstation layout for local models - nanobrook - 10-15-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at NVFP4. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Best workstation layout for local models - vramarch - 10-18-2026
Can you fill in a bit more context? Is this local-only or are you routing anything externally?
Re: Best workstation layout for local models - mlxgarden - 10-18-2026
Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?
Re: Best workstation layout for local models - vramghost - 10-19-2026
Not sure I agree there. I get why that seems right, but on 7900 XTX the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.