+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Proprietary / Frontier (https://www.datasculpting.ai/inference/forumdisplay.php?fid=48)
+--- Thread: best local open-weight model this week (/showthread.php?tid=913)
Pages:
1
2
best local open-weight model this week - circuitwire58 - 10-08-2026
Did a bit of testing and thought I'd post the result here.
Short version: LM Studio behaved well for me, but only after I stopped ignoring speculative decoding.
Nothing scientific here, just practical notes from a real setup.
Re: best local open-weight model this week - sparrowghost - 10-08-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at continuous batching. Once you know the plain path works, then add the faster path back in piece by piece.
Re: best local open-weight model this week - bytestack - 10-08-2026
Not sure I agree there. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.
Re: best local open-weight model this week - amberplane - 10-08-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at workstation thermals.
Re: best local open-weight model this week - sparrowshed - 10-11-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there.
Re: best local open-weight model this week - amberkite - 10-11-2026
Thanks, that helps. That matches what I saw on 7900 XTX. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: best local open-weight model this week - mellowstream - 10-11-2026
Tried to reproduce it just now. On A6000 with TensorRT-LLM, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.
Re: best local open-weight model this week - bytestack - 10-11-2026
Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?
Re: best local open-weight model this week - thetaember38 - 10-14-2026
worth checking the driver first.
Re: best local open-weight model this week - mellowstream - 10-14-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.