prefix caching in llama.cpp with PyTorch

[visionyard23]

I think the bottleneck is somewhere else. I've seen people blame KV cache pressure when the real issue was much more mundane. Could be a workload difference, but I wouldn't generalize too fast from one setup.