+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Context / KV Cache (https://www.datasculpting.ai/inference/forumdisplay.php?fid=41)
+--- Thread: prefix caching on repeated workflows (/showthread.php?tid=42)
prefix caching on repeated workflows - cacheline - 08-26-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Maybe useful as a starting point: vLLM on dual 3090, keeping INT4 in mind from the start.
Feel free to tear it apart if you see something dumb.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.