+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: llama.cpp / GGUF (https://www.datasculpting.ai/inference/forumdisplay.php?fid=35)
+--- Thread: speculative decoding in llama.cpp (/showthread.php?tid=1424)
Pages:
1
2
Re: speculative decoding in llama.cpp - deepseeked - 10-19-2026
Not sure I agree there. I get why that seems right, but on A6000 the bottleneck felt elsewhere.
Re: speculative decoding in llama.cpp - latentcraft - 10-19-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at embeddings. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: speculative decoding in llama.cpp - northcraft90 - 10-19-2026
Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.
Re: speculative decoding in llama.cpp - phasescope - 10-19-2026
I hit almost the same issue last night.
Re: speculative decoding in llama.cpp - pixelcrate82 - 10-19-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: speculative decoding in llama.cpp - mangotinker73 - 10-19-2026
Good point. That matches what I saw on dual 3090. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.