+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Edge AI & Gadgets (https://www.datasculpting.ai/inference/forumdisplay.php?fid=21)
+--- Forum: Mobile (https://www.datasculpting.ai/inference/forumdisplay.php?fid=73)
+--- Thread: mobile NPU reality check (/showthread.php?tid=601)
Pages:
1
2
mobile NPU reality check - fastquill74 - 10-01-2026
Posting this in case it saves someone else a little time.
I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about quant drift.
Simple and boring is a valid answer.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Re: mobile NPU reality check - llmpeak - 10-01-2026
Quote:Posting this in case it saves someone else a little time.
worth checking the driver first.
Re: mobile NPU reality check - torchkite - 10-02-2026
Quote:Quote:Posting this in case it saves someone else a little time.
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. I expected the model choice to matter more than it did.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: mobile NPU reality check - plainscope50 - 10-03-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at small local stacks. Not glamorous advice, but repeatable baselines save a lot of time.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: mobile NPU reality check - sparrowworks88 - 10-03-2026
Quote:What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at small local stacks. N
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there.
Re: mobile NPU reality check - relaysupply90 - 10-03-2026
same here on my side.
Re: mobile NPU reality check - rocmriley - 10-04-2026
Quote:same here on my side.
I think the bottleneck is somewhere else. I get why that seems right, but on mini PC the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: mobile NPU reality check - marbleyard32 - 10-04-2026
Appreciate the detail. That matches what I saw on B200. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
I'm happy to test one more variant if you want a second data point.
Re: mobile NPU reality check - routeline23 - 10-05-2026
Quote:Appreciate the detail. That matches what I saw on B200. Worth rechecking your defaults though, because one bad assumption can make the rest of the thr
subscribed, curious where this lands.
I'm happy to test one more variant if you want a second data point.
Re: mobile NPU reality check - sparrowghost - 10-05-2026
That matches what I was seeing. I ended up in about the same place using vLLM. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.