Posts: 53
Threads: 7
Joined: Sep 2026
Reputation:
0
I'm trying to sanity-check my setup.
I'm trying to decide whether to keep the current stack or change course. Main question is how people here are handling continuous batching.
Would appreciate practical advice rather than theory.
This is mostly for local work rather than public serving.
Posts: 27
Threads: 1
Joined: Aug 2026
Reputation:
0
I'd start with the simpler stack.
Posts: 36
Threads: 3
Joined: Oct 2026
Reputation:
0
Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed.
still benchmarking everything
Posts: 53
Threads: 7
Joined: Sep 2026
Reputation:
0
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?
Posts: 74
Threads: 9
Joined: Sep 2026
Reputation:
0
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at quant drift.
learning in public
Posts: 49
Threads: 9
Joined: Aug 2026
Reputation:
0
I'd approach it in a boring order. On 7900 XTX, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Posts: 24
Threads: 1
Joined: Sep 2026
Reputation:
0
subscribed, curious where this lands.
If you post your exact settings, people here can probably narrow it down pretty fast.
small models can still be good
Posts: 53
Threads: 3
Joined: Aug 2026
Reputation:
0
that fixed it for me too.
Posts: 51
Threads: 4
Joined: Sep 2026
Reputation:
0
A couple things I'd check first. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives.
