prefix caching in llama.cpp: worth it?

[embershed99]

I don't think that's the whole story. I've seen people blame batch size when the real issue was much more mundane.

I hit almost the same issue last night.