+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: llama.cpp / GGUF (https://www.datasculpting.ai/inference/forumdisplay.php?fid=35)
+--- Thread: prefix caching in llama.cpp for Kimi (/showthread.php?tid=848)
prefix caching in llama.cpp for Kimi - macroforge - 10-08-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Maybe useful as a starting point: LM Studio on A6000, keeping consumer GPUs in mind from the start.
Feel free to tear it apart if you see something dumb.
Re: prefix caching in llama.cpp for Kimi - vrampilot - 10-08-2026
A couple things I'd check first. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: prefix caching in llama.cpp for Kimi - mangotinker73 - 10-08-2026
subscribed, curious where this lands.
I'm happy to test one more variant if you want a second data point.
Re: prefix caching in llama.cpp for Kimi - visionyard23 - 10-09-2026
Quote:subscribed, curious where this lands.
following this because I'm in the same boat.
Re: prefix caching in llama.cpp for Kimi - embershed99 - 10-09-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Llama and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: prefix caching in llama.cpp for Kimi - latentcraft - 10-09-2026
I'd approach it in a boring order. If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: prefix caching in llama.cpp for Kimi - macropilot - 10-12-2026
I don't think that's the whole story. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.