+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Edge AI & Gadgets (https://www.datasculpting.ai/inference/forumdisplay.php?fid=21)
+--- Forum: SBCs / Mini PCs (https://www.datasculpting.ai/inference/forumdisplay.php?fid=74)
+--- Thread: SBC inference experiments on GB200 (/showthread.php?tid=156)
SBC inference experiments on GB200 - mintquill - 09-06-2026
Posting this in case it saves someone else a little time.
I'm not trying to build a perfect architecture diagram. I just want a sane starting point around PyTorch and Threadripper box.
Simple and boring is a valid answer.
This is mostly for local work rather than public serving.
Re: SBC inference experiments on GB200 - homelabhenry - 09-08-2026
worth checking the driver first.
Re: SBC inference experiments on GB200 - thetatinker77 - 09-11-2026
Thanks, that helps. That matches what I saw on dual 3090.
Re: SBC inference experiments on GB200 - cobalttinker - 09-11-2026
That matches what I was seeing. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: SBC inference experiments on GB200 - vramghost - 09-13-2026
Quote:That matches what I was seeing. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at wh
My first guess would be this: On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: SBC inference experiments on GB200 - lablab - 09-13-2026
A couple things I'd check first. On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: SBC inference experiments on GB200 - homelabhenry - 09-13-2026
Not sure I agree there. I've seen people blame consumer GPUs when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.
Re: SBC inference experiments on GB200 - thetatinker77 - 09-14-2026
Appreciate the detail. The pattern around token cost has been pretty similar for me.
Re: SBC inference experiments on GB200 - homelabhenry - 09-14-2026
that fixed it for me too.