+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Other Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=39)
+--- Thread: KV cache compression worth it (/showthread.php?tid=400)
Pages:
1
2
Re: KV cache compression worth it - kernelmike - 09-27-2026
Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.
Re: KV cache compression worth it - vramstream - 09-28-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?