+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Edge AI & Gadgets (https://www.datasculpting.ai/inference/forumdisplay.php?fid=21)
+--- Forum: SBCs / Mini PCs (https://www.datasculpting.ai/inference/forumdisplay.php?fid=74)
+--- Thread: SBC inference experiments: early impressions (/showthread.php?tid=410)
Pages:
1
2
Re: SBC inference experiments: early impressions - homelabhenry - 10-04-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at KV cache pressure. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: SBC inference experiments: early impressions - cobalttinker - 10-04-2026
that fixed it for me too.
Re: SBC inference experiments: early impressions - homelabhenry - 10-04-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tokens/sec. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: SBC inference experiments: early impressions - quantsupply - 10-05-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at workstation thermals.
Re: SBC inference experiments: early impressions - promptfield - 10-05-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at workstation thermals. Once you know the plain path works, then add the faster path back in piece by piece.