+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Prompting & Context Engineering (https://www.datasculpting.ai/inference/forumdisplay.php?fid=15)
+--- Forum: Prompts / System Prompts (https://www.datasculpting.ai/inference/forumdisplay.php?fid=62)
+--- Thread: prompt templates worth keeping for Llama (/showthread.php?tid=890)
prompt templates worth keeping for Llama - quiettoken - 10-07-2026
This keeps coming up in conversation so I figured I'd start a thread.
One thing I keep noticing is that people underestimate prefix caching until they actually try to run this stuff for more than one afternoon.
Wondering if this is just where the whole space is heading now.
This is mostly for local work rather than public serving.
Re: prompt templates worth keeping for Llama - sysstation69 - 10-09-2026
Maybe, but I had the opposite result. I've seen people blame tokens/sec when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one.
Re: prompt templates worth keeping for Llama - thetafield - 10-09-2026
My first guess would be this: On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.