+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Apps / Utilities / Tools (https://www.datasculpting.ai/inference/forumdisplay.php?fid=18)
+--- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=20)
+--- Thread: desktop apps around local AI: setup advice (/showthread.php?tid=508)
Pages:
1
2
Re: desktop apps around local AI: setup advice - audioriver - 10-07-2026
Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.
Re: desktop apps around local AI: setup advice - batchshed - 10-07-2026
subscribed, curious where this lands.
Re: desktop apps around local AI: setup advice - edgecoast - 10-07-2026
Maybe, but I had the opposite result. I've seen people blame KV cache pressure when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.
Re: desktop apps around local AI: setup advice - llmpeak - 10-07-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there.
Re: desktop apps around local AI: setup advice - modgarden - 10-07-2026
Thanks, that helps. That matches what I saw on M3 Max. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.