|
speculative decoding in llama.cpp
|
10-06-2026, 11:49 AM
Quote:Tried to reproduce it just now. On B200 with vLLM, I saw basically the same behavior. The surprise was that memory pressure showed up before throughpu What worked for me was pretty simple. On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives. |
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
speculative decoding in llama.cpp - by vrambandit - 10-06-2026, 06:29 AM
Re: speculative decoding in llama.cpp - by embershed99 - 10-06-2026, 08:43 AM
Re: speculative decoding in llama.cpp - by vrambandit - 10-06-2026, 10:49 AM
Re: speculative decoding in llama.cpp - by vramridge - 10-06-2026, 11:49 AM
Re: speculative decoding in llama.cpp - by macroforge - 10-06-2026, 08:28 PM
Re: speculative decoding in llama.cpp - by syshouse - 10-06-2026, 08:50 PM
Re: speculative decoding in llama.cpp - by macroforge - 10-06-2026, 10:24 PM
Re: speculative decoding in llama.cpp - by nanopatch - 10-06-2026, 11:30 PM
Re: speculative decoding in llama.cpp - by vrampilot - 10-08-2026, 12:20 AM
Re: speculative decoding in llama.cpp - by mangotinker73 - 10-08-2026, 12:47 AM
Re: speculative decoding in llama.cpp - by northcraft90 - 10-08-2026, 09:13 AM
|
