+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: Systems & Builds (https://www.datasculpting.ai/inference/forumdisplay.php?fid=29)
+--- Thread: Threadripper build for inference: setup advice (/showthread.php?tid=10)
Threadripper build for inference: setup advice - homelabhenry - 08-22-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Maybe useful as a starting point: LM Studio on dual 3090, keeping prefill/decode split in mind from the start.
Maybe useful as a starting point for newer folks.
Re: Threadripper build for inference: setup advice - latentpeak22 - 08-23-2026
Can you fill in a bit more context? Is this local-only or are you routing anything externally? I keep promising myself this is the last rebuild.
Re: Threadripper build for inference: setup advice - batchwire - 08-23-2026
Quote:Can you fill in a bit more context? Is this local-only or are you routing anything externally? I keep promising myself this is the last rebuild.
Appreciate the detail. That matches what I saw on 5090.
Re: Threadripper build for inference: setup advice - phasenode - 08-24-2026
I think the bottleneck is somewhere else. I get why that seems right, but on mini PC the bottleneck felt elsewhere.
Re: Threadripper build for inference: setup advice - vectorcrate98 - 08-24-2026
That matches what I was seeing. That matches what I saw on dual 3090.
Re: Threadripper build for inference: setup advice - labnest - 08-24-2026
That matches what I was seeing. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: Threadripper build for inference: setup advice - homelabhenry - 08-27-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at speculative decoding. Not glamorous advice, but repeatable baselines save a lot of time.
Re: Threadripper build for inference: setup advice - macroforge - 08-27-2026
Thanks, that helps. I ended up in about the same place using SGLang. The biggest change for me was simplifying the stack instead of adding one more optimization layer.