+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Context / KV Cache (https://www.datasculpting.ai/inference/forumdisplay.php?fid=41)
+--- Thread: speculative decoding actually worth it (/showthread.php?tid=158)
Pages:
1
2
Re: speculative decoding actually worth it - cacheline - 09-12-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: speculative decoding actually worth it - northhall88 - 09-13-2026
I hit almost the same issue last night.