rackmount vs tower for homelab inference: setup advice

[batchwire]

Can you fill in a bit more context? Are you testing this on GB200 or something else?

[tensorport]

same here on my side.

[Data Sculpting]

Maybe, but I had the opposite result. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.

[cozygrid12]

Interesting, I hadn't tried that. The pattern around prefill/decode split has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.