desktop apps around local AI: setup advice

[audioriver]

Did a small comparison this afternoon. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.

[batchshed]

subscribed, curious where this lands.

[edgecoast]

Maybe, but I had the opposite result. I've seen people blame KV cache pressure when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.

[llmpeak]

My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there.

[modgarden]

Thanks, that helps. That matches what I saw on M3 Max. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.