speculative decoding in llama.cpp

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Linear Mode

speculative decoding in llama.cpp

mangotinker73 Offline

Junior Member

Posts: 43
Threads: 5
Joined: Sep 2026
Reputation: 0

09-20-2026, 04:29 PM

Did a small comparison this afternoon. On B200 with vLLM, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.

Find

« Next Oldest | Next Newest »

Messages In This Thread

speculative decoding in llama.cpp - by mangoquill - 09-14-2026, 03:43 PM

Re: speculative decoding in llama.cpp - by embershed99 - 09-15-2026, 04:23 PM

Re: speculative decoding in llama.cpp - by oxidestream - 09-17-2026, 01:50 PM

Re: speculative decoding in llama.cpp - by macroforge - 09-17-2026, 04:43 AM

Re: speculative decoding in llama.cpp - by kernelgrid - 09-17-2026, 08:49 AM

Re: speculative decoding in llama.cpp - by bytesupply76 - 09-17-2026, 09:45 AM

Re: speculative decoding in llama.cpp - by dockridge - 09-17-2026, 03:02 PM

Re: speculative decoding in llama.cpp - by sparrowworks88 - 09-20-2026, 03:53 PM

Re: speculative decoding in llama.cpp - by bytesupply76 - 09-26-2026, 06:09 AM

Re: speculative decoding in llama.cpp - by mangotinker73 - 09-20-2026, 04:29 PM

Re: speculative decoding in llama.cpp - by embershed99 - 09-23-2026, 04:57 PM

Re: speculative decoding in llama.cpp - by northcraft90 - 09-24-2026, 05:11 AM

Re: speculative decoding in llama.cpp - by vrambandit - 09-24-2026, 05:28 PM

Re: speculative decoding in llama.cpp - by pixelcrate82 - 09-26-2026, 10:37 AM

View a Printable Version

Forum Jump:

Users browsing this thread: 1 Guest(s)

Login
Username:
Password:	Lost Password?
	Remember me

Sign in with X.com