+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: Systems & Builds (https://www.datasculpting.ai/inference/forumdisplay.php?fid=29)
+--- Thread: Best workstation layout for local models (/showthread.php?tid=666)
Best workstation layout for local models - stackpilot - 10-02-2026
Posting this in case it saves someone else a little time.
I'm not trying to build a perfect architecture diagram. I just want a sane starting point around MLX and 7900 XTX.
Simple and boring is a valid answer.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Re: Best workstation layout for local models - labnest - 10-04-2026
Appreciate the detail. I ended up in about the same place using vLLM. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: Best workstation layout for local models - torchkite - 10-04-2026
I hit almost the same issue last night.
I'm happy to test one more variant if you want a second data point.
Re: Best workstation layout for local models - homelabhenry - 10-05-2026
I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: Best workstation layout for local models - bytegarden73 - 10-05-2026
I'd approach it in a boring order. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Best workstation layout for local models - rasterpoint - 10-05-2026
Quote:I'd approach it in a boring order. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. O
I hit almost the same issue last night.
Re: Best workstation layout for local models - batchwire - 10-05-2026
Quote:[quote]I'd approach it in a boring order. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obv
Thanks, that helps. The pattern around reranking has been pretty similar for me.
Re: Best workstation layout for local models - sanevault - 10-06-2026
My first guess would be this: On Threadripper box, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: Best workstation layout for local models - batchwire - 10-06-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly.