VRAM saving for image generation: what are you seeing?

[dockember]

Maybe I'm missing something obvious.

I'm mostly working with FLUX in SGLang on A6000. The part I'm trying to understand is KV cache pressure.

If you've tested something similar, what did you keep and what did you stop bothering with? The logs were somehow calmer than I was.

[pixelcrate82]

Thanks, that helps. That matches what I saw on M3 Max. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[deepseeked]

Interesting, I hadn't tried that. That matches what I saw on Threadripper box.

[ggufwire22]

I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.

[amberplane]

Ran a quick check on my box. On 7900 XTX with Ollama, I saw basically the same behavior.

[signaltrack44]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Llama and compare from there.

following this because I'm in the same boat.

[emberquill]

Good point. That matches what I saw on A6000.

[amberkite]

A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?

[ggufwire22]

Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed.

[nanogarden]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Llama and compare from there.