speculative decoding in llama.cpp with SGLang

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Linear Mode

speculative decoding in llama.cpp with SGLang

quantsupply Offline

Offline

Junior Member

Posts: 34
Threads: 4
Joined: Aug 2026
Reputation: 0

#8

10-11-2026, 04:42 AM

Maybe, but I had the opposite result. I've seen people blame prefix caching when the real issue was much more mundane.

small models can still be good

Find

Reply

« Next Oldest | Next Newest »

Messages In This Thread

speculative decoding in llama.cpp with SGLang - by embershed99 - 10-06-2026, 08:49 PM

Re: speculative decoding in llama.cpp with SGLang - by pixelcrate82 - 10-07-2026, 09:15 AM

Re: speculative decoding in llama.cpp with SGLang - by batchcraft - 10-10-2026, 09:35 AM

Re: speculative decoding in llama.cpp with SGLang - by macroforge - 10-10-2026, 09:53 PM

Re: speculative decoding in llama.cpp with SGLang - by dockridge - 10-10-2026, 10:12 PM

Re: speculative decoding in llama.cpp with SGLang - by mangotinker73 - 10-11-2026, 02:56 AM

Re: speculative decoding in llama.cpp with SGLang - by mellowwire98 - 10-11-2026, 03:49 AM

Re: speculative decoding in llama.cpp with SGLang - by quantsupply - 10-11-2026, 04:42 AM

Re: speculative decoding in llama.cpp with SGLang - by pixelcrate82 - 10-11-2026, 05:34 AM

Re: speculative decoding in llama.cpp with SGLang - by northcraft90 - 10-11-2026, 06:18 AM

Re: speculative decoding in llama.cpp with SGLang - by mellowwire98 - 10-11-2026, 06:56 AM

Re: speculative decoding in llama.cpp with SGLang - by kernelghost - 10-11-2026, 07:54 AM

Re: speculative decoding in llama.cpp with SGLang - by macroforge - 10-12-2026, 08:44 PM

View a Printable Version

Forum Jump:

Users browsing this thread: 1 Guest(s)