video model setup pain points

[mintquill]

Posting this in case it saves someone else a little time.

I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about consumer GPUs.

Would rather learn from your scars than collect my own all at once.

[paperrunner]

that fixed it for me too.

[quietcraft]

A couple questions before I guess too hard. Is this local-only or are you routing anything externally?

[cudapeak]

I don't think that's the whole story. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.

[tensorriver]

A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?

[torchstation]

A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?

Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.

If you post your exact settings, people here can probably narrow it down pretty fast.

[oxidegrove40]

A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[kernelgrid]

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at small local stacks. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[bytegarden73]

Appreciate the detail. That matches what I saw on 7900 XTX. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[vrambandit]

Appreciate the detail. The pattern around continuous batching has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.