+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: Other Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=39)
+--- Thread: KV cache compression worth it for Mixtral (/showthread.php?tid=851)
KV cache compression worth it for Mixtral - deltagrid - 10-07-2026
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once VRAM headroom becomes important.
Trying to avoid rebuilding the whole thing twice.