image generation model locals actually use

[cudapeak]

Spent some time on this today and wanted to compare notes.

Two paths both seem viable, but they break down differently once embeddings becomes important.

What would you pick if you had to live with it for a few months?

[benchplease]

Spent some time on this today and wanted to compare notes.

Maybe, but I had the opposite result. I've seen people blame consumer GPUs when the real issue was much more mundane.

[fastlab]

Good point. That matches what I saw on mini PC. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[sparrowhall]

Good point. That matches what I saw on mini PC. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at NVFP4.

[vramstream]

What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there.

[vrambandit]

What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with DeepSeek and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.

[plainscope50]

I think the bottleneck is somewhere else. I get why that seems right, but on M3 Max the bottleneck felt elsewhere. Worth testing the simpler explanation before adopting the more interesting one.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.

[southtinker]

I think the bottleneck is somewhere else. I get why that seems right, but on M3 Max the bottleneck felt elsewhere. Worth testing the simpler explanati

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at consumer GPUs. Once you know the plain path works, then add the faster path back in piece by piece.

[sparrowcraft]

What worked for me was pretty simple. On GB200, the easiest win is often to back off the aggressive settings and see whether the problem survives.