+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: Systems & Builds (https://www.datasculpting.ai/inference/forumdisplay.php?fid=29)
+--- Thread: Best workstation layout for local models: anyone testing this? (/showthread.php?tid=1028)
Pages:
1
2
Best workstation layout for local models: anyone testing this? - batchwire - 10-10-2026
Has anyone else run into this?
I'm mostly working with FLUX in MLX on 5090. The part I'm trying to understand is workstation thermals.
Would appreciate practical advice rather than theory.
Re: Best workstation layout for local models: anyone testing this? - stackpilot - 10-10-2026
subscribed, curious where this lands.
Re: Best workstation layout for local models: anyone testing this? - tensorworks - 10-10-2026
Quote:subscribed, curious where this lands.
same here on my side. At this point the real bottleneck is my patience.
Re: Best workstation layout for local models: anyone testing this? - sanevault - 10-10-2026
What worked for me was pretty simple. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time. The logs were somehow calmer than I was.
Re: Best workstation layout for local models: anyone testing this? - fastscope - 10-11-2026
Can you fill in a bit more context? Are you testing this on A6000 or something else?
Re: Best workstation layout for local models: anyone testing this? - plainscope50 - 10-11-2026
Not sure I agree there. I've seen people blame small local stacks when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: Best workstation layout for local models: anyone testing this? - homelabhenry - 10-12-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at prefill/decode split. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Best workstation layout for local models: anyone testing this? - vramghost - 10-12-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Llama and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: Best workstation layout for local models: anyone testing this? - modbench - 10-13-2026
following this because I'm in the same boat.
Re: Best workstation layout for local models: anyone testing this? - bytegarden73 - 10-13-2026
Quote:following this because I'm in the same boat.
Thanks, that helps. The pattern around continuous batching has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.