+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Context / KV Cache (https://www.datasculpting.ai/inference/forumdisplay.php?fid=41)
+--- Thread: speculative decoding actually worth it with LM Studio (/showthread.php?tid=908)
Pages:
1
2
Re: speculative decoding actually worth it with LM Studio - cacheline - 10-13-2026
Quote:I'd approach it in a boring order. On 4090, the easiest win is often to back off the aggressive settings and see whether the problem survives.
My first guess would be this: On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: speculative decoding actually worth it with LM Studio - audiodesk - 10-14-2026
I don't think that's the whole story. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: speculative decoding actually worth it with LM Studio - cacheline - 10-14-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
Re: speculative decoding actually worth it with LM Studio - latentcraft - 10-14-2026
Quote:What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. Not glamorous advic
I tested this tonight. On Threadripper box with PyTorch, I saw basically the same behavior. I expected the model choice to matter more than it did.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.