context size vs speed in GGUF with vLLM

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Linear Mode

context size vs speed in GGUF with vLLM

audioriver Offline

Member

Posts: 69
Threads: 3
Joined: Sep 2026
Reputation: 0

10-16-2026, 05:45 AM

Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.

still benchmarking everything

Find

« Next Oldest | Next Newest »

Messages In This Thread

context size vs speed in GGUF with vLLM - by embershed99 - 10-12-2026, 01:53 AM

Re: context size vs speed in GGUF with vLLM - by vrambandit - 10-13-2026, 02:33 AM

Re: context size vs speed in GGUF with vLLM - by audioriver - 10-16-2026, 05:45 AM

Re: context size vs speed in GGUF with vLLM - by vectorport - 10-16-2026, 02:58 AM

Re: context size vs speed in GGUF with vLLM - by mangotinker73 - 10-18-2026, 06:51 PM

Re: context size vs speed in GGUF with vLLM - by mellowwire98 - 10-18-2026, 08:17 PM

Re: context size vs speed in GGUF with vLLM - by pixelcrate82 - 10-18-2026, 08:57 PM

Re: context size vs speed in GGUF with vLLM - by latentcraft - 10-18-2026, 10:38 PM

Re: context size vs speed in GGUF with vLLM - by cudacrate - 10-18-2026, 10:06 PM

Re: context size vs speed in GGUF with vLLM - by bytesupply76 - 10-18-2026, 10:51 PM

Re: context size vs speed in GGUF with vLLM - by mangotinker73 - 10-18-2026, 10:12 PM

Re: context size vs speed in GGUF with vLLM - by vrambandit - 10-19-2026, 02:29 AM

Re: context size vs speed in GGUF with vLLM - by macroforge - 10-19-2026, 03:37 AM

Re: context size vs speed in GGUF with vLLM - by phasescope - 10-19-2026, 04:06 AM

View a Printable Version

Forum Jump:

Users browsing this thread: 1 Guest(s)

Login
Username:
Password:	Lost Password?
	Remember me

Sign in with X.com