speculative decoding in llama.cpp with SGLang

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Linear Mode

speculative decoding in llama.cpp with SGLang

northcraft90 Offline

Offline

Member

Posts: 63
Threads: 7
Joined: Sep 2026
Reputation: 0

#10

10-11-2026, 06:18 AM

Ran a quick check on my box. On B200 with MLX, I saw basically the same behavior. The main thing that moved was latency, not quality.

Find

Reply

« Next Oldest | Next Newest »

Messages In This Thread

speculative decoding in llama.cpp with SGLang - by embershed99 - 10-06-2026, 08:49 PM

Re: speculative decoding in llama.cpp with SGLang - by pixelcrate82 - 10-07-2026, 09:15 AM

Re: speculative decoding in llama.cpp with SGLang - by batchcraft - 10-10-2026, 09:35 AM

Re: speculative decoding in llama.cpp with SGLang - by macroforge - 10-10-2026, 09:53 PM

Re: speculative decoding in llama.cpp with SGLang - by dockridge - 10-10-2026, 10:12 PM

Re: speculative decoding in llama.cpp with SGLang - by mangotinker73 - 10-11-2026, 02:56 AM

Re: speculative decoding in llama.cpp with SGLang - by mellowwire98 - 10-11-2026, 03:49 AM

Re: speculative decoding in llama.cpp with SGLang - by quantsupply - 10-11-2026, 04:42 AM

Re: speculative decoding in llama.cpp with SGLang - by pixelcrate82 - 10-11-2026, 05:34 AM

Re: speculative decoding in llama.cpp with SGLang - by northcraft90 - 10-11-2026, 06:18 AM

Re: speculative decoding in llama.cpp with SGLang - by mellowwire98 - 10-11-2026, 06:56 AM

Re: speculative decoding in llama.cpp with SGLang - by kernelghost - 10-11-2026, 07:54 AM

Re: speculative decoding in llama.cpp with SGLang - by macroforge - 10-12-2026, 08:44 PM

View a Printable Version

Forum Jump:

Users browsing this thread: 1 Guest(s)