context size vs speed in GGUF: worth it?

[northcraft90]

Did a bit of testing and thought I'd post the result here.

After a few runs on 7900 XTX, I ended up with decent performance, but the setup only really made sense once I adjusted around tokens/sec.

Curious if anyone else landed in the same place.

[vectorport]

A couple questions before I guess too hard. Are you testing this on 5090 or something else?

[vrambandit]

My first guess would be this: On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.