+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: Systems & Builds (https://www.datasculpting.ai/inference/forumdisplay.php?fid=29)
+--- Thread: rackmount vs tower for homelab inference on dual 3090 (/showthread.php?tid=185)
rackmount vs tower for homelab inference on dual 3090 - batchyard - 09-08-2026
Has anyone else run into this?
I'm trying to decide whether to keep the current stack or change course. Main question is how people here are handling batch size.
If you've tested something similar, what did you keep and what did you stop bothering with?
Re: rackmount vs tower for homelab inference on dual 3090 - tensorport - 09-08-2026
Can you fill in a bit more context? Are you testing this on B200 or something else?
Re: rackmount vs tower for homelab inference on dual 3090 - vramghost - 09-09-2026
I tested this tonight. My result was mixed: first run looked bad, second run looked normal. I expected the model choice to matter more than it did.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: rackmount vs tower for homelab inference on dual 3090 - phasenode - 09-09-2026
following this because I'm in the same boat.
Re: rackmount vs tower for homelab inference on dual 3090 - stackpilot - 09-09-2026
I don't think that's the whole story. I've seen people blame reranking when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: rackmount vs tower for homelab inference on dual 3090 - tensorport - 09-10-2026
Interesting, I hadn't tried that. That matches what I saw on Threadripper box.
Re: rackmount vs tower for homelab inference on dual 3090 - batchwire - 09-11-2026
A couple things I'd check first. If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: rackmount vs tower for homelab inference on dual 3090 - cobaltlab - 09-11-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there.
Re: rackmount vs tower for homelab inference on dual 3090 - homelabhenry - 09-12-2026
I hit almost the same issue last night.
Re: rackmount vs tower for homelab inference on dual 3090 - tensorworks - 09-13-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.