|
speculative decoding in llama.cpp
|
10-06-2026, 10:11 PM
Quote:Ran a quick check on my box. On A6000 with PyTorch, I saw basically the same behavior. I expected the model choice to matter more than it did. Thanks, that helps. I ended up in about the same place using TensorRT-LLM. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing. |
|
« Next Oldest | Next Newest »
|
| Messages In This Thread |
|
speculative decoding in llama.cpp - by latentcraft - 10-04-2026, 09:32 PM
Re: speculative decoding in llama.cpp - by dockworks23 - 10-05-2026, 09:55 PM
Re: speculative decoding in llama.cpp - by mangotinker73 - 10-06-2026, 10:11 PM
Re: speculative decoding in llama.cpp - by thetagrid15 - 10-07-2026, 12:22 PM
Re: speculative decoding in llama.cpp - by vrambandit - 10-07-2026, 11:01 AM
Re: speculative decoding in llama.cpp - by northcraft90 - 10-08-2026, 01:38 AM
Re: speculative decoding in llama.cpp - by macroforge - 10-08-2026, 12:44 AM
Re: speculative decoding in llama.cpp - by plaingarden18 - 10-08-2026, 01:47 AM
Re: speculative decoding in llama.cpp - by dockridge - 10-08-2026, 02:24 AM
|
