image generation model locals actually use: what are you seeing?

[signalcraft14]

Spent some time on this today and wanted to compare notes.

I'm comparing two approaches here: keep it simple on B200, or squeeze more out of the stack and deal with the complexity.

Trying to avoid rebuilding the whole thing twice.

For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.

[bytelab]

A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[fastquill74]

A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. A lot of these threads end

Good point. The pattern around NVFP4 has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

I'm happy to test one more variant if you want a second data point.

[labnest]

I'd approach it in a boring order. If you're on MLX, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.

[cudapeak]

I hit almost the same issue last night.

[modbench]

Interesting, I hadn't tried that. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[paperrunner]

Interesting, I hadn't tried that. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at

I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The surprise was that memory pressure showed up before throughput collapsed.

[southpatch]

Interesting, I hadn't tried that. That matches what I saw on 5090. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[kernelpeak]

Interesting, I hadn't tried that. That matches what I saw on 5090. The biggest change for me was simplifying the stack instead of adding one more opti

A couple things I'd check first. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious.

[macroforge]

[quote]Interesting, I hadn't tried that. That matches what I saw on 5090. The biggest change for me was simplifying the stack instead of adding one mo

Can you fill in a bit more context? What model, what runtime, and roughly what context size?

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.