speculative decoding in llama.cpp with SGLang

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Linear Mode

speculative decoding in llama.cpp with SGLang

macroforge Offline

Junior Member

Posts: 43
Threads: 3
Joined: Sep 2026
Reputation: 0

#13

10-12-2026, 08:44 PM

Appreciate the detail. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

mostly local

Find

« Next Oldest | Next Newest »

Messages In This Thread

speculative decoding in llama.cpp with SGLang - by embershed99 - 10-06-2026, 08:49 PM

Re: speculative decoding in llama.cpp with SGLang - by pixelcrate82 - 10-07-2026, 09:15 AM

Re: speculative decoding in llama.cpp with SGLang - by batchcraft - 10-10-2026, 09:35 AM

Re: speculative decoding in llama.cpp with SGLang - by macroforge - 10-10-2026, 09:53 PM

Re: speculative decoding in llama.cpp with SGLang - by dockridge - 10-10-2026, 10:12 PM

Re: speculative decoding in llama.cpp with SGLang - by mangotinker73 - 10-11-2026, 02:56 AM

Re: speculative decoding in llama.cpp with SGLang - by mellowwire98 - 10-11-2026, 03:49 AM

Re: speculative decoding in llama.cpp with SGLang - by quantsupply - 10-11-2026, 04:42 AM

Re: speculative decoding in llama.cpp with SGLang - by pixelcrate82 - 10-11-2026, 05:34 AM

Re: speculative decoding in llama.cpp with SGLang - by northcraft90 - 10-11-2026, 06:18 AM

Re: speculative decoding in llama.cpp with SGLang - by mellowwire98 - 10-11-2026, 06:56 AM

Re: speculative decoding in llama.cpp with SGLang - by kernelghost - 10-11-2026, 07:54 AM

Re: speculative decoding in llama.cpp with SGLang - by macroforge - 10-12-2026, 08:44 PM

View a Printable Version

Forum Jump:

Users browsing this thread: 1 Guest(s)

Login
Username:
Password:	Lost Password?
	Remember me

Sign in with X.com