video model setup pain points on GB200

[routecraft]

Good point. That matches what I saw on mini PC. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[mangotinker73]

Thanks, that helps. I ended up in about the same place using MLX. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.

[llmarch51]

I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at continuous batching.

[cozygrid12]

Not sure I agree there. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Worth testing the simpler explanation before adopting the more interesting one.