+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: llama.cpp / GGUF (https://www.datasculpting.ai/inference/forumdisplay.php?fid=35)
+--- Thread: context size vs speed in GGUF: what are you seeing? (/showthread.php?tid=1280)
context size vs speed in GGUF: what are you seeing? - torchsupply99 - 10-15-2026
Spent some time on this today and wanted to compare notes.
I'm comparing two approaches here: keep it simple on mini PC, or squeeze more out of the stack and deal with the complexity.
Interested in tradeoffs more than fan loyalty.
Re: context size vs speed in GGUF: what are you seeing? - papermixer4 - 10-16-2026
Quote:Spent some time on this today and wanted to compare notes.
Thanks, that helps. The pattern around continuous batching has been pretty similar for me.
Re: context size vs speed in GGUF: what are you seeing? - vrambandit - 10-17-2026
Quote:Quote:Spent some time on this today and wanted to compare notes.
I think the bottleneck is somewhere else. I've seen people blame router policy when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.