prefix caching in llama.cpp: worth it?

[phasescope]

This keeps coming up in conversation so I figured I'd start a thread.

A couple things I'd check first. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious.