+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: Systems & Builds (https://www.datasculpting.ai/inference/forumdisplay.php?fid=29)
+--- Thread: Best workstation layout for local models (/showthread.php?tid=125)
Pages:
1
2
Best workstation layout for local models - latentpeak22 - 09-03-2026
Spent some time on this today and wanted to compare notes.
I've gone back and forth between MLX and llama.cpp for FLUX. The interesting difference for me has been consumer GPUs.
Trying to avoid rebuilding the whole thing twice.
Re: Best workstation layout for local models - rocmriley - 09-04-2026
Quote:Spent some time on this today and wanted to compare notes.
same here on my side. The logs were somehow calmer than I was.
Re: Best workstation layout for local models - sparrowworks88 - 09-04-2026
Quote:Quote:Spent some time on this today and wanted to compare notes.
Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?
Re: Best workstation layout for local models - torchkite - 09-04-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: Best workstation layout for local models - vramstream - 09-05-2026
worth checking the driver first.
Re: Best workstation layout for local models - tensorworks - 09-05-2026
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.
Re: Best workstation layout for local models - vramghost - 09-08-2026
Quote:Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, no
I'd start with the simpler stack.
Re: Best workstation layout for local models - kernelhouse70 - 09-08-2026
Tried to reproduce it just now. On 7900 XTX with SGLang, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.
Re: Best workstation layout for local models - vectorcrate98 - 09-10-2026
I don't think that's the whole story. I get why that seems right, but on mini PC the bottleneck felt elsewhere.
Re: Best workstation layout for local models - tensorport - 09-13-2026
Quote:I don't think that's the whole story. I get why that seems right, but on mini PC the bottleneck felt elsewhere.
I tested this tonight. On GB200 with LM Studio, I saw basically the same behavior.