+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: llama.cpp / GGUF (https://www.datasculpting.ai/inference/forumdisplay.php?fid=35)
+--- Thread: prefix caching in llama.cpp (/showthread.php?tid=1252)
prefix caching in llama.cpp - mellowwire98 - 10-14-2026
This keeps coming up in conversation so I figured I'd start a thread.
The more I work with local setups, the more it seems like workstation thermals decides everything downstream.
Wondering if this is just where the whole space is heading now.
Re: prefix caching in llama.cpp - thetagrid15 - 10-14-2026
I'd approach it in a boring order. If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.