+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: News & Industry (https://www.datasculpting.ai/inference/forumdisplay.php?fid=26)
+--- Thread: latest release actually worth attention with PyTorch (/showthread.php?tid=391)
Pages:
1
2
latest release actually worth attention with PyTorch - fastlab - 09-20-2026
Curious what people here are seeing.
My current setup is PyTorch with 5090, and I keep running into tool calling.
Would appreciate practical advice rather than theory.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: latest release actually worth attention with PyTorch - audiopoint18 - 09-20-2026
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?
Re: latest release actually worth attention with PyTorch - oxidestream - 09-22-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at HBM constraints.
Re: latest release actually worth attention with PyTorch - dockember - 09-22-2026
My first guess would be this: If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: latest release actually worth attention with PyTorch - deepseeked - 09-22-2026
Maybe, but I had the opposite result. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.
Re: latest release actually worth attention with PyTorch - thetashed24 - 09-23-2026
I'd start with the simpler stack.
Re: latest release actually worth attention with PyTorch - dockriver - 09-24-2026
A couple things I'd check first. If you're on MLX, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
Re: latest release actually worth attention with PyTorch - fastlab - 09-24-2026
Quote:A couple things I'd check first. If you're on MLX, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you kno
What worked for me was pretty simple. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: latest release actually worth attention with PyTorch - mangowire - 09-24-2026
What exact setup are you running? What model, what runtime, and roughly what context size?
Re: latest release actually worth attention with PyTorch - thetashed24 - 09-24-2026
I'd start with the simpler stack.