+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: Systems & Builds (https://www.datasculpting.ai/inference/forumdisplay.php?fid=29)
+--- Thread: Best workstation layout for local models (/showthread.php?tid=420)
Best workstation layout for local models - kernelhouse70 - 09-23-2026
Posting this in case it saves someone else a little time.
I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about MoE serving.
Would rather learn from your scars than collect my own all at once.
Re: Best workstation layout for local models - edgequill - 09-23-2026
My first guess would be this: If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Best workstation layout for local models - signaltrack44 - 09-23-2026
A couple things I'd check first. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Best workstation layout for local models - sanevault - 09-23-2026
Quote:A couple things I'd check first. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once
Interesting, I hadn't tried that. The pattern around long context has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: Best workstation layout for local models - nanobrook - 09-23-2026
That matches what I was seeing. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: Best workstation layout for local models - phasenode - 09-23-2026
Thanks, that helps. The pattern around quant drift has been pretty similar for me.
Re: Best workstation layout for local models - vramghost - 09-23-2026
Not sure I agree there. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Could be a workload difference, but I wouldn't generalize too fast from one setup.
I'm happy to test one more variant if you want a second data point.
Re: Best workstation layout for local models - fastbench - 09-23-2026
What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?