speculative decoding in llama.cpp with SGLang

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Linear Mode

speculative decoding in llama.cpp with SGLang

mellowwire98 Offline

Offline

Member

Posts: 60
Threads: 9
Joined: Aug 2026
Reputation: 0

#11

10-11-2026, 06:56 AM

Interesting, I hadn't tried that. The pattern around continuous batching has been pretty similar for me.

Find

Reply

« Next Oldest | Next Newest »

Messages In This Thread

speculative decoding in llama.cpp with SGLang - by embershed99 - 10-06-2026, 08:49 PM

Re: speculative decoding in llama.cpp with SGLang - by pixelcrate82 - 10-07-2026, 09:15 AM

Re: speculative decoding in llama.cpp with SGLang - by batchcraft - 10-10-2026, 09:35 AM

Re: speculative decoding in llama.cpp with SGLang - by macroforge - 10-10-2026, 09:53 PM

Re: speculative decoding in llama.cpp with SGLang - by dockridge - 10-10-2026, 10:12 PM

Re: speculative decoding in llama.cpp with SGLang - by mangotinker73 - 10-11-2026, 02:56 AM

Re: speculative decoding in llama.cpp with SGLang - by mellowwire98 - 10-11-2026, 03:49 AM

Re: speculative decoding in llama.cpp with SGLang - by quantsupply - 10-11-2026, 04:42 AM

Re: speculative decoding in llama.cpp with SGLang - by pixelcrate82 - 10-11-2026, 05:34 AM

Re: speculative decoding in llama.cpp with SGLang - by northcraft90 - 10-11-2026, 06:18 AM

Re: speculative decoding in llama.cpp with SGLang - by mellowwire98 - 10-11-2026, 06:56 AM

Re: speculative decoding in llama.cpp with SGLang - by kernelghost - 10-11-2026, 07:54 AM

Re: speculative decoding in llama.cpp with SGLang - by macroforge - 10-12-2026, 08:44 PM

View a Printable Version

Forum Jump:

Users browsing this thread: 1 Guest(s)