prefix caching in llama.cpp: worth it?

[latentcraft]

I hit almost the same issue last night.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.