+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: llama.cpp / GGUF (https://www.datasculpting.ai/inference/forumdisplay.php?fid=35)
+--- Thread: context size vs speed in GGUF with vLLM (/showthread.php?tid=1045)
Pages:
1
2
Re: context size vs speed in GGUF with vLLM - bytesupply76 - 10-18-2026
same here on my side.
Re: context size vs speed in GGUF with vLLM - vrambandit - 10-19-2026
Not sure I agree there. I get why that seems right, but on 7900 XTX the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: context size vs speed in GGUF with vLLM - macroforge - 10-19-2026
Good point. The pattern around reranking has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
I'm happy to test one more variant if you want a second data point.
Re: context size vs speed in GGUF with vLLM - phasescope - 10-19-2026
Quote:Good point. The pattern around reranking has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make th
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at KV cache pressure.