+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: llama.cpp / GGUF (https://www.datasculpting.ai/inference/forumdisplay.php?fid=35)
+--- Thread: context size vs speed in GGUF: worth it? (/showthread.php?tid=338)
context size vs speed in GGUF: worth it? - northcraft90 - 09-17-2026
Did a bit of testing and thought I'd post the result here.
After a few runs on 7900 XTX, I ended up with decent performance, but the setup only really made sense once I adjusted around tokens/sec.
Curious if anyone else landed in the same place.
Re: context size vs speed in GGUF: worth it? - vectorport - 09-19-2026
A couple questions before I guess too hard. Are you testing this on 5090 or something else?
Re: context size vs speed in GGUF: worth it? - vrambandit - 09-20-2026
My first guess would be this: On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.