Posts: 42
Threads: 10
Joined: Aug 2026
Reputation:
0
This keeps coming up in conversation so I figured I'd start a thread.
It feels like the conversation has shifted hard toward inference and serving details lately, and honestly that makes sense once you start dealing with workstation thermals.
Curious whether that lines up with what other people are seeing.
Posts: 27
Threads: 3
Joined: Sep 2026
Reputation:
0
Appreciate the detail. I ended up in about the same place using PyTorch.
mostly local
Posts: 134
Threads: 18
Joined: Sep 2026
Reputation:
0
Maybe, but I had the opposite result. I've seen people blame batch size when the real issue was much more mundane.
watching release notes so you don't have to
Posts: 43
Threads: 6
Joined: Aug 2026
Reputation:
0
subscribed, curious where this lands.
learning in public
Posts: 29
Threads: 6
Joined: Sep 2026
Reputation:
0
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher.
still benchmarking everything
Posts: 40
Threads: 1
Joined: Sep 2026
Reputation:
0
That matches what I was seeing. That matches what I saw on A6000.
learning in public
Posts: 59
Threads: 7
Joined: Oct 2026
Reputation:
0
Appreciate the detail. That matches what I saw on A6000. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
learning in public
Posts: 44
Threads: 8
Joined: Oct 2026
Reputation:
0
A couple things I'd check first. On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Posts: 40
Threads: 1
Joined: Sep 2026
Reputation:
0
What exact setup are you running? What model, what runtime, and roughly what context size?
learning in public
