+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Evaluation & Testing (https://www.datasculpting.ai/inference/forumdisplay.php?fid=16)
+--- Forum: Model Comparisons (https://www.datasculpting.ai/inference/forumdisplay.php?fid=67)
+--- Thread: VRAM saving for image generation for FLUX (/showthread.php?tid=100)
Pages:
1
2
VRAM saving for image generation for FLUX - rocmrunner40 - 09-01-2026
Spent some time on this today and wanted to compare notes.
I've gone back and forth between LM Studio and SGLang for Llama. The interesting difference for me has been NVFP4.
Trying to avoid rebuilding the whole thing twice.
For context, I'm not chasing leaderboard numbers. I mostly want the system to feel consistent day to day.
Re: VRAM saving for image generation for FLUX - nanopatch - 09-01-2026
That matches what I was seeing. The pattern around VRAM headroom has been pretty similar for me.
Re: VRAM saving for image generation for FLUX - vectorpoint - 09-02-2026
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?
Re: VRAM saving for image generation for FLUX - fastbench - 09-02-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: VRAM saving for image generation for FLUX - thetapath - 09-05-2026
Quote:What exact setup are you running? Is this local-only or are you routing anything externally?
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there.
Re: VRAM saving for image generation for FLUX - turboarch95 - 09-05-2026
I'd start with the simpler stack.
Re: VRAM saving for image generation for FLUX - batchfield55 - 09-05-2026
Quote:I'd start with the simpler stack.
Interesting, I hadn't tried that. The pattern around continuous batching has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
I'm happy to test one more variant if you want a second data point.
Re: VRAM saving for image generation for FLUX - kernelrunner - 09-05-2026
Good point. That matches what I saw on dual 3090. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: VRAM saving for image generation for FLUX - thetarunner - 09-05-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
I'm happy to test one more variant if you want a second data point.
Re: VRAM saving for image generation for FLUX - cozyarch - 09-05-2026
Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.