+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Edge AI & Gadgets (https://www.datasculpting.ai/inference/forumdisplay.php?fid=21)
+--- Forum: Mobile (https://www.datasculpting.ai/inference/forumdisplay.php?fid=73)
+--- Thread: running local models on a phone: what are you seeing? (/showthread.php?tid=521)
running local models on a phone: what are you seeing? - promptspark58 - 09-26-2026
Posting this in case it saves someone else a little time.
I'm not trying to build a perfect architecture diagram. I just want a sane starting point around PyTorch and 4090.
Simple and boring is a valid answer.
Re: running local models on a phone: what are you seeing? - cozysmith - 09-26-2026
Quote:Posting this in case it saves someone else a little time.
Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal.
Re: running local models on a phone: what are you seeing? - orbitline - 09-27-2026
Quote:Quote:Posting this in case it saves someone else a little time.
I tested this tonight. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.
Re: running local models on a phone: what are you seeing? - sparrowshed - 09-28-2026
Good point. The pattern around workstation thermals has been pretty similar for me.
Re: running local models on a phone: what are you seeing? - torchkite - 09-29-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: running local models on a phone: what are you seeing? - cudacrate - 09-29-2026
What exact setup are you running? Is this local-only or are you routing anything externally?