+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Edge AI & Gadgets (https://www.datasculpting.ai/inference/forumdisplay.php?fid=21)
+--- Forum: AI Devices (https://www.datasculpting.ai/inference/forumdisplay.php?fid=75)
+--- Thread: dedicated AI gadgets you actually use (/showthread.php?tid=959)
Pages:
1
2
Re: dedicated AI gadgets you actually use - turboscope - 10-18-2026
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?
Re: dedicated AI gadgets you actually use - bytepilot - 10-18-2026
Thanks, that helps. The pattern around VRAM headroom has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: dedicated AI gadgets you actually use - flintriver50 - 10-18-2026
I'd approach it in a boring order. If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: dedicated AI gadgets you actually use - mintquill - 10-19-2026
Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. I expected the model choice to matter more than it did.
Re: dedicated AI gadgets you actually use - routecrate - 10-19-2026
What exact setup are you running? Are you testing this on mini PC or something else?
Re: dedicated AI gadgets you actually use - rasterbench - 10-19-2026
Quote:A couple questions before I guess too hard. What model, what runtime, and roughly what context size?
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there.