prefix caching in llama.cpp for Phi

[latentcraft]

[quote]Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not

I hit almost the same issue last night.

I'm happy to test one more variant if you want a second data point.