10-18-2026, 08:26 PM
Quote:I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
Thanks, that helps. The pattern around KV cache pressure has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
2x3090 • llama.cpp • too much VRAM is still not enough