Best workstation layout for local models: anyone testing this?

[batchwire]

Has anyone else run into this?

I'm mostly working with FLUX in MLX on 5090. The part I'm trying to understand is workstation thermals.

Would appreciate practical advice rather than theory.

[stackpilot]

subscribed, curious where this lands.

[tensorworks]

subscribed, curious where this lands.

same here on my side. At this point the real bottleneck is my patience.

[sanevault]

What worked for me was pretty simple. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time. The logs were somehow calmer than I was.

[fastscope]

Can you fill in a bit more context? Are you testing this on A6000 or something else?

[plainscope50]

Not sure I agree there. I've seen people blame small local stacks when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.

[homelabhenry]

I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at prefill/decode split. Once you know the plain path works, then add the faster path back in piece by piece.

[vramghost]

A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Llama and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.

[modbench]

following this because I'm in the same boat.

[bytegarden73]

following this because I'm in the same boat.

Thanks, that helps. The pattern around continuous batching has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.