speculative decoding in llama.cpp

[mangotinker73]

Good point. That matches what I saw on dual 3090. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.