+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Edge AI & Gadgets (https://www.datasculpting.ai/inference/forumdisplay.php?fid=21)
+--- Forum: Mobile (https://www.datasculpting.ai/inference/forumdisplay.php?fid=73)
+--- Thread: on-device inference worth the effort: what are you seeing? (/showthread.php?tid=1075)
Pages:
1
2
Re: on-device inference worth the effort: what are you seeing? - batchscope72 - 10-18-2026
Appreciate the detail. The pattern around consumer GPUs has been pretty similar for me.
Re: on-device inference worth the effort: what are you seeing? - fusedriver - 10-18-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: on-device inference worth the effort: what are you seeing? - mellowstream - 10-18-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: on-device inference worth the effort: what are you seeing? - relaysupply90 - 10-18-2026
Quote:Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Appreciate the detail. That matches what I saw on 7900 XTX. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: on-device inference worth the effort: what are you seeing? - cozysmith - 10-19-2026
Can you fill in a bit more context? Are you testing this on GB200 or something else?
Re: on-device inference worth the effort: what are you seeing? - thetaember38 - 10-19-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there.