Posts: 38
Threads: 7
Joined: Oct 2026
Reputation:
0
Spent some time on this today and wanted to compare notes.
Two paths both seem viable, but they break down differently once VRAM headroom becomes important.
Interested in tradeoffs more than fan loyalty.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Posts: 36
Threads: 7
Joined: Sep 2026
Reputation:
0
I'd approach it in a boring order. On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Posts: 112
Threads: 12
Joined: Sep 2026
Reputation:
0
A couple things I'd check first. On 4090, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
If you post your exact settings, people here can probably narrow it down pretty fast.
context windows are not a product strategy
Posts: 60
Threads: 5
Joined: Sep 2026
Reputation:
0
Not sure I agree there. I've seen people blame tokens/sec when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Posts: 49
Threads: 7
Joined: Oct 2026
Reputation:
0
Appreciate the detail. I ended up in about the same place using TensorRT-LLM. I keep promising myself this is the last rebuild.
trying not to rebuild again
Posts: 60
Threads: 5
Joined: Sep 2026
Reputation:
0
What worked for me was pretty simple. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Posts: 36
Threads: 7
Joined: Sep 2026
Reputation:
0
I think the bottleneck is somewhere else. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere.
Posts: 112
Threads: 12
Joined: Sep 2026
Reputation:
0
Quote:I think the bottleneck is somewhere else. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere.
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.
context windows are not a product strategy
Posts: 58
Threads: 8
Joined: Sep 2026
Reputation:
0
Quote:Quote:I think the bottleneck is somewhere else. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere.
Interesting, I hadn't tried that. I ended up in about the same place using MLX.
trying not to rebuild again
Posts: 60
Threads: 5
Joined: Sep 2026
Reputation:
0
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tool calling. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
