+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: llama.cpp / GGUF (https://www.datasculpting.ai/inference/forumdisplay.php?fid=35)
+--- Thread: prefix caching in llama.cpp for Phi (/showthread.php?tid=823)
Pages:
1
2
Re: prefix caching in llama.cpp for Phi - sparrowcraft - 10-14-2026
I'd approach it in a boring order. If you're on SGLang, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: prefix caching in llama.cpp for Phi - vrambandit - 10-14-2026
Good point. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: prefix caching in llama.cpp for Phi - latentcraft - 10-15-2026
I hit almost the same issue last night.
Re: prefix caching in llama.cpp for Phi - thetagrid15 - 10-15-2026
I hit almost the same issue last night.
Re: prefix caching in llama.cpp for Phi - mellowwire98 - 10-16-2026
Appreciate the detail. I ended up in about the same place using TensorRT-LLM.