+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: News & Industry (https://www.datasculpting.ai/inference/forumdisplay.php?fid=26)
+--- Thread: driver/framework news roundup on 4090 (/showthread.php?tid=1384)
Pages:
1
2
driver/framework news roundup on 4090 - southpatch - 10-17-2026
Posting this in case it saves someone else a little time.
I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about prefill/decode split.
I'm especially interested in mistakes worth avoiding.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: driver/framework news roundup on 4090 - deepseeked - 10-18-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Phi and compare from there.
Re: driver/framework news roundup on 4090 - fastlab - 10-18-2026
That matches what I was seeing. The pattern around speculative decoding has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: driver/framework news roundup on 4090 - dockriver - 10-18-2026
Interesting, I hadn't tried that. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: driver/framework news roundup on 4090 - deepseeked - 10-18-2026
Tried to reproduce it just now. On mini PC with Ollama, I saw basically the same behavior.
Re: driver/framework news roundup on 4090 - localpatch78 - 10-18-2026
A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?
Re: driver/framework news roundup on 4090 - deepseeked - 10-18-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there.
Re: driver/framework news roundup on 4090 - audiopoint18 - 10-19-2026
I'd approach it in a boring order. If you're on PyTorch, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
Re: driver/framework news roundup on 4090 - thetashed24 - 10-19-2026
I hit almost the same issue last night.
Re: driver/framework news roundup on 4090 - deepseeked - 10-19-2026
What exact setup are you running? What model, what runtime, and roughly what context size?