+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: CPUs / NPUs / Accelerators (https://www.datasculpting.ai/inference/forumdisplay.php?fid=30)
+--- Thread: consumer accelerators worth watching with Ollama (/showthread.php?tid=1178)
Pages:
1
2
Re: consumer accelerators worth watching with Ollama - mlxgarden - 10-19-2026
Quote:Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at small local stacks.
Re: consumer accelerators worth watching with Ollama - rocmcrate26 - 10-19-2026
Quote:[quote]Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not qu
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?