Posts: 53
Threads: 7
Joined: Sep 2026
Reputation:
0
Maybe I'm missing something obvious.
I'm trying to decide whether to keep the current stack or change course. Main question is how people here are handling tokens/sec.
If you've tested something similar, what did you keep and what did you stop bothering with?
I care more about stability than squeezing the last possible tokens/sec out of it.
Posts: 57
Threads: 8
Joined: Oct 2026
Reputation:
0
Thanks, that helps. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Posts: 39
Threads: 1
Joined: Sep 2026
Reputation:
0
My first guess would be this: If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
queue depth watcher
Posts: 45
Threads: 6
Joined: Aug 2026
Reputation:
0
Thanks, that helps. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
mostly local
Posts: 137
Threads: 16
Joined: Sep 2026
Reputation:
0
A couple things I'd check first. On dual 3090, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
throughput matters, but tail latency matters too
Posts: 57
Threads: 8
Joined: Oct 2026
Reputation:
0
I'd approach it in a boring order. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Posts: 63
Threads: 5
Joined: Sep 2026
Reputation:
0
Can you fill in a bit more context? Do you see the same thing with a smaller batch or shorter prompt?
