+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Context / KV Cache (https://www.datasculpting.ai/inference/forumdisplay.php?fid=41)
+--- Thread: prefix caching on repeated workflows (/showthread.php?tid=1076)
Pages:
1
2
Re: prefix caching on repeated workflows - cacheline - 10-19-2026
That matches what I was seeing. I ended up in about the same place using MLX. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: prefix caching on repeated workflows - marbletrack - 10-19-2026
Thanks, that helps. The pattern around tokens/sec has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: prefix caching on repeated workflows - cacheline - 10-19-2026
My first guess would be this: If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: prefix caching on repeated workflows - southspark - 10-19-2026
Good point. I don't think your conclusion is far off.
Re: prefix caching on repeated workflows - macropilot - 10-19-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tokens/sec.