KV cache compression worth it for Mistral

[kernelghost]

Interesting, I hadn't tried that. I ended up in about the same place using SGLang. The biggest change for me was simplifying the stack instead of adding one more optimization layer. The logs were somehow calmer than I was.