+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: llama.cpp / GGUF (https://www.datasculpting.ai/inference/forumdisplay.php?fid=35)
+--- Thread: prefix caching in llama.cpp (/showthread.php?tid=1391)
Pages:
1
2
Re: prefix caching in llama.cpp - vrambandit - 10-19-2026
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?
Re: prefix caching in llama.cpp - mellowwire98 - 10-19-2026
Maybe, but I had the opposite result. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
Re: prefix caching in llama.cpp - latentcraft - 10-19-2026
A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.