context size vs speed in GGUF with vLLM

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Linear Mode

context size vs speed in GGUF with vLLM

macroforge Offline

Offline

Junior Member

Posts: 43
Threads: 3
Joined: Sep 2026
Reputation: 0

#13

10-19-2026, 03:37 AM

Good point. The pattern around reranking has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

I'm happy to test one more variant if you want a second data point.

mostly local

Find

Reply

« Next Oldest | Next Newest »

Messages In This Thread

context size vs speed in GGUF with vLLM - by embershed99 - 10-12-2026, 01:53 AM

Re: context size vs speed in GGUF with vLLM - by vrambandit - 10-13-2026, 02:33 AM

Re: context size vs speed in GGUF with vLLM - by audioriver - 10-16-2026, 05:45 AM

Re: context size vs speed in GGUF with vLLM - by vectorport - 10-16-2026, 02:58 AM

Re: context size vs speed in GGUF with vLLM - by mangotinker73 - 10-18-2026, 06:51 PM

Re: context size vs speed in GGUF with vLLM - by mellowwire98 - 10-18-2026, 08:17 PM

Re: context size vs speed in GGUF with vLLM - by pixelcrate82 - 10-18-2026, 08:57 PM

Re: context size vs speed in GGUF with vLLM - by latentcraft - 10-18-2026, 10:38 PM

Re: context size vs speed in GGUF with vLLM - by cudacrate - 10-18-2026, 10:06 PM

Re: context size vs speed in GGUF with vLLM - by bytesupply76 - 10-18-2026, 10:51 PM

Re: context size vs speed in GGUF with vLLM - by mangotinker73 - 10-18-2026, 10:12 PM

Re: context size vs speed in GGUF with vLLM - by vrambandit - 10-19-2026, 02:29 AM

Re: context size vs speed in GGUF with vLLM - by macroforge - 10-19-2026, 03:37 AM

Re: context size vs speed in GGUF with vLLM - by phasescope - 10-19-2026, 04:06 AM

View a Printable Version

Forum Jump:

Users browsing this thread: 1 Guest(s)