+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: News & Industry (https://www.datasculpting.ai/inference/forumdisplay.php?fid=26)
+--- Thread: driver/framework news roundup (/showthread.php?tid=1305)
Pages:
1
2
driver/framework news roundup - modbench - 10-15-2026
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once quant drift becomes important.
Trying to avoid rebuilding the whole thing twice.
I expected the biggest difference to come from the model, but the stack around it mattered at least as much.
Re: driver/framework news roundup - phasescope - 10-18-2026
Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?
Re: driver/framework news roundup - batchfield55 - 10-18-2026
Quote:Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at VRAM headroom. Not glamorous advice, but repeatable baselines save a lot of time.
Re: driver/framework news roundup - dockbrook21 - 10-18-2026
Thanks, that helps. That matches what I saw on 4090. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: driver/framework news roundup - phaseshed - 10-18-2026
That matches what I was seeing. That matches what I saw on dual 3090.
Re: driver/framework news roundup - dockriver - 10-19-2026
Need a little more detail here. Is this local-only or are you routing anything externally?
Re: driver/framework news roundup - routearch - 10-19-2026
What exact setup are you running? What model, what runtime, and roughly what context size?
Re: driver/framework news roundup - amberrunner - 10-19-2026
Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet. At this point the real bottleneck is my patience.
Re: driver/framework news roundup - modbench - 10-19-2026
Maybe, but I had the opposite result. I've seen people blame MoE serving when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: driver/framework news roundup - deepseeked - 10-19-2026
worth checking the driver first.