+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Security, Privacy & Safety (https://www.datasculpting.ai/inference/forumdisplay.php?fid=23)
+--- Forum: Local Privacy (https://www.datasculpting.ai/inference/forumdisplay.php?fid=76)
+--- Thread: what truly stays local (/showthread.php?tid=480)
what truly stays local - modgarden - 09-25-2026
I figured I'd ask here before changing everything.
I'm mostly working with Llama in Ollama on 7900 XTX. The part I'm trying to understand is workstation thermals.
Not looking for the perfect answer, mostly what worked in real use.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Re: what truly stays local - rocmhouse90 - 09-26-2026
Thanks, that helps. The pattern around NVFP4 has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: what truly stays local - llmpeak - 09-26-2026
Interesting, I hadn't tried that. The pattern around prefill/decode split has been pretty similar for me.
Re: what truly stays local - northhall88 - 09-28-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at NVFP4.
Re: what truly stays local - audioriver - 09-28-2026
I'd approach it in a boring order. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: what truly stays local - mangoquill - 09-28-2026
Maybe, but I had the opposite result. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: what truly stays local - modgarden - 09-28-2026
A couple things I'd check first. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.