speculative decoding in llama.cpp

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Linear Mode

speculative decoding in llama.cpp

vrambandit Offline

Member

Posts: 168
Threads: 14
Joined: Sep 2026
Reputation: 0

09-23-2026, 12:35 AM

Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.

2x3090 • llama.cpp • too much VRAM is still not enough

Find

« Next Oldest | Next Newest »

Messages In This Thread

speculative decoding in llama.cpp - by quantghost - 09-20-2026, 11:48 PM

Re: speculative decoding in llama.cpp - by vrambandit - 09-23-2026, 12:35 AM

Re: speculative decoding in llama.cpp - by vectorport - 09-24-2026, 01:01 PM

Re: speculative decoding in llama.cpp - by macroforge - 09-25-2026, 01:39 PM

Re: speculative decoding in llama.cpp - by orbitgarden - 09-28-2026, 01:49 PM

Re: speculative decoding in llama.cpp - by vrambandit - 09-30-2026, 02:40 AM

Re: speculative decoding in llama.cpp - by bytegarden73 - 10-01-2026, 06:06 PM

Re: speculative decoding in llama.cpp - by latentcraft - 09-30-2026, 07:03 AM

Re: speculative decoding in llama.cpp - by northcraft90 - 09-30-2026, 07:19 AM

Re: speculative decoding in llama.cpp - by vrambandit - 09-30-2026, 07:35 AM

Re: speculative decoding in llama.cpp - by pixelcrate82 - 09-30-2026, 07:45 AM

Re: speculative decoding in llama.cpp - by dockridge - 10-02-2026, 06:26 AM

Re: speculative decoding in llama.cpp - by vrambandit - 09-30-2026, 04:23 PM

Re: speculative decoding in llama.cpp - by northcraft90 - 09-30-2026, 04:34 PM

Re: speculative decoding in llama.cpp - by vrambandit - 10-01-2026, 04:38 AM

Re: speculative decoding in llama.cpp - by thetagrid15 - 10-01-2026, 05:56 AM

View a Printable Version

Forum Jump:

Users browsing this thread: 1 Guest(s)

Login
Username:
Password:	Lost Password?
	Remember me

Sign in with X.com