+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: Systems & Builds (https://www.datasculpting.ai/inference/forumdisplay.php?fid=29)
+--- Thread: rackmount vs tower for homelab inference (/showthread.php?tid=1128)
rackmount vs tower for homelab inference - latentpeak22 - 10-12-2026
Spent some time on this today and wanted to compare notes.
I've gone back and forth between MLX and Ollama for Llama. The interesting difference for me has been HBM constraints.
What would you pick if you had to live with it for a few months?
Re: rackmount vs tower for homelab inference - rocmcrate26 - 10-12-2026
A couple questions before I guess too hard. Are you testing this on GB200 or something else?
Re: rackmount vs tower for homelab inference - cozygrid12 - 10-12-2026
Ran a quick check on my box. On M3 Max with SGLang, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.
Re: rackmount vs tower for homelab inference - tensorport - 10-12-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: rackmount vs tower for homelab inference - homelabhenry - 10-13-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it. The logs were somehow calmer than I was.
Re: rackmount vs tower for homelab inference - tensorport - 10-13-2026
Not sure I agree there. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.
Re: rackmount vs tower for homelab inference - vramghost - 10-13-2026
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
I'm happy to test one more variant if you want a second data point.
Re: rackmount vs tower for homelab inference - rocmriley - 10-13-2026
That matches what I was seeing. That matches what I saw on dual 3090. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.