+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: llama.cpp / GGUF (https://www.datasculpting.ai/inference/forumdisplay.php?fid=35)
+--- Thread: speculative decoding in llama.cpp on dual 3090 (/showthread.php?tid=1293)
speculative decoding in llama.cpp on dual 3090 - vectorport - 10-16-2026
I figured I'd ask here before changing everything.
I'm mostly working with FLUX in PyTorch on A6000. The part I'm trying to understand is HBM constraints.
Would appreciate practical advice rather than theory.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: speculative decoding in llama.cpp on dual 3090 - rasterbench - 10-16-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at consumer GPUs.
Re: speculative decoding in llama.cpp on dual 3090 - pixelcrate82 - 10-16-2026
That matches what I was seeing. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: speculative decoding in llama.cpp on dual 3090 - visionforge - 10-18-2026
That matches what I was seeing. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: speculative decoding in llama.cpp on dual 3090 - mellowwire98 - 10-18-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Llama and compare from there.
Re: speculative decoding in llama.cpp on dual 3090 - vrambandit - 10-18-2026
Need a little more detail here. Is this local-only or are you routing anything externally?
Re: speculative decoding in llama.cpp on dual 3090 - mangotinker73 - 10-19-2026
Can you fill in a bit more context? Are you testing this on B200 or something else?