prefix caching on repeated workflows on 7900 XTX

[cacheline]

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at consumer GPUs. Once you know

I'd approach it in a boring order. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[dockember]

What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Gemma and compare from there.

[cacheline]

That matches what I was seeing. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[marbleyard32]

subscribed, curious where this lands.

[cacheline]

Appreciate the detail. That matches what I saw on dual 3090. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.