|
speculative decoding in llama.cpp
|
10-06-2026, 08:28 PM
My first guess would be this: On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
If you post your exact settings, people here can probably narrow it down pretty fast.
mostly local
|
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
speculative decoding in llama.cpp - by vrambandit - 10-06-2026, 06:29 AM
Re: speculative decoding in llama.cpp - by embershed99 - 10-06-2026, 08:43 AM
Re: speculative decoding in llama.cpp - by vrambandit - 10-06-2026, 10:49 AM
Re: speculative decoding in llama.cpp - by vramridge - 10-06-2026, 11:49 AM
Re: speculative decoding in llama.cpp - by macroforge - 10-06-2026, 08:28 PM
Re: speculative decoding in llama.cpp - by syshouse - 10-06-2026, 08:50 PM
Re: speculative decoding in llama.cpp - by macroforge - 10-06-2026, 10:24 PM
Re: speculative decoding in llama.cpp - by nanopatch - 10-06-2026, 11:30 PM
Re: speculative decoding in llama.cpp - by vrampilot - 10-08-2026, 12:20 AM
Re: speculative decoding in llama.cpp - by mangotinker73 - 10-08-2026, 12:47 AM
Re: speculative decoding in llama.cpp - by northcraft90 - 10-08-2026, 09:13 AM
|
