running local models on a phone for Mistral - Printable Version +- Inference (https://www.datasculpting.ai/inference) +-- Forum: Edge AI & Gadgets (https://www.datasculpting.ai/inference/forumdisplay.php?fid=21) +--- Forum: Mobile (https://www.datasculpting.ai/inference/forumdisplay.php?fid=73) +--- Thread: running local models on a phone for Mistral (/showthread.php?tid=594)

running local models on a phone for Mistral - fusedriver - 09-30-2026 Did a bit of testing and thought I'd post the result here. I tried this with Llama using PyTorch on mini PC. The headline result was better than I expected, but consumer GPUs still ended up being the limiting factor. Nothing scientific here, just practical notes from a real setup. Re: running local models on a phone for Mistral - emberquill - 09-30-2026 Thanks, that helps. I ended up in about the same place using SGLang.