prefix caching in llama.cpp

[mellowwire98]

This keeps coming up in conversation so I figured I'd start a thread.

The more I work with local setups, the more it seems like workstation thermals decides everything downstream.

Wondering if this is just where the whole space is heading now.