when video is still not worth the wait: anyone testing this?

[fastquill74]

Thought I'd post a small recipe because a lot of threads ask similar things.

Maybe useful as a starting point: Ollama on 4090, keeping tail latency in mind from the start.

Maybe useful as a starting point for newer folks.

I care more about stability than squeezing the last possible tokens/sec out of it.

I expected the biggest difference to come from the model, but the stack around it mattered at least as much.

[plainworks]

Can you fill in a bit more context? Is this local-only or are you routing anything externally?

[mintstream9]

Can you fill in a bit more context? Is this local-only or are you routing anything externally?

Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.

[vectorhouse71]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[audionode8]

Good point. That matches what I saw on 5090.

[batchyard]

A couple things I'd check first. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[deltastation]

A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious.

[paperrunner]

I'd approach it in a boring order. On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.

[vectorpoint]

Not sure I agree there. I get why that seems right, but on GB200 the bottleneck felt elsewhere.

[circuitworks90]

I'd start with the simpler stack.