+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: llama.cpp / GGUF (https://www.datasculpting.ai/inference/forumdisplay.php?fid=35)
+--- Thread: context size vs speed in GGUF: worth it? (/showthread.php?tid=1487)
Pages:
1
2
Re: context size vs speed in GGUF: worth it? - pixelcrate82 - 10-19-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: context size vs speed in GGUF: worth it? - thetarunner - 10-19-2026
That matches what I was seeing. I ended up in about the same place using llama.cpp. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: context size vs speed in GGUF: worth it? - routeforge64 - 10-19-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at consumer GPUs.
Re: context size vs speed in GGUF: worth it? - phasescope - 10-19-2026
That matches what I was seeing. I ended up in about the same place using Ollama.
Re: context size vs speed in GGUF: worth it? - thetagrid15 - 10-19-2026
A couple things I'd check first. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: context size vs speed in GGUF: worth it? - macroforge - 10-19-2026
Thanks, that helps. I ended up in about the same place using SGLang. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.