+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: News & Industry (https://www.datasculpting.ai/inference/forumdisplay.php?fid=26)
+--- Thread: industry move that matters for local users (/showthread.php?tid=1122)
industry move that matters for local users - latentpeak22 - 10-13-2026
This keeps coming up in conversation so I figured I'd start a thread.
It feels like the conversation has shifted hard toward inference and serving details lately, and honestly that makes sense once you start dealing with workstation thermals.
Curious whether that lines up with what other people are seeing.
Re: industry move that matters for local users - mangopeak72 - 10-14-2026
Appreciate the detail. I ended up in about the same place using PyTorch.
Re: industry move that matters for local users - deepseeked - 10-16-2026
Maybe, but I had the opposite result. I've seen people blame batch size when the real issue was much more mundane.
Re: industry move that matters for local users - nanogarden - 10-16-2026
subscribed, curious where this lands.
Re: industry move that matters for local users - mangowire - 10-16-2026
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher.
Re: industry move that matters for local users - vectorpoint - 10-16-2026
That matches what I was seeing. That matches what I saw on A6000.
Re: industry move that matters for local users - thetashed24 - 10-18-2026
Appreciate the detail. That matches what I saw on A6000. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: industry move that matters for local users - fastlab - 10-18-2026
A couple things I'd check first. On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: industry move that matters for local users - vectorpoint - 10-18-2026
What exact setup are you running? What model, what runtime, and roughly what context size?