+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Edge AI & Gadgets (https://www.datasculpting.ai/inference/forumdisplay.php?fid=21)
+--- Forum: Mobile (https://www.datasculpting.ai/inference/forumdisplay.php?fid=73)
+--- Thread: running local models on a phone (/showthread.php?tid=223)
Pages:
1
2
Re: running local models on a phone - rocmplane - 09-15-2026
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?
Re: running local models on a phone - routeline23 - 09-16-2026
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.
If you post your exact settings, people here can probably narrow it down pretty fast.