VRAM saving for image generation: what are you seeing?

[edgekite]

Appreciate the detail. The pattern around KV cache pressure has been pretty similar for me.

[relayforge]

A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?

What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?

[cobaltlab]

Appreciate the detail. The pattern around KV cache pressure has been pretty similar for me.

that fixed it for me too.

[sparrowkite53]

Appreciate the detail. The pattern around KV cache pressure has been pretty similar for me.

I'd start with the simpler stack.

[batchfield55]

Thanks, that helps. The pattern around token cost has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[sanescope]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.