+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Inference Engines (https://www.datasculpting.ai/inference/forumdisplay.php?fid=9)
+--- Forum: TensorRT-LLM (https://www.datasculpting.ai/inference/forumdisplay.php?fid=37)
+--- Thread: serving on Blackwell for FLUX (/showthread.php?tid=92)
Pages:
1
2
serving on Blackwell for FLUX - kernelmike - 08-31-2026
Did a bit of testing and thought I'd post the result here.
Short version: LM Studio behaved well for me, but only after I stopped ignoring VRAM headroom.
Nothing scientific here, just practical notes from a real setup.
Re: serving on Blackwell for FLUX - audioforge - 08-31-2026
Thanks, that helps. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
I'm happy to test one more variant if you want a second data point.
Re: serving on Blackwell for FLUX - pixelcrate82 - 08-31-2026
Quote:Thanks, that helps. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimizat
Maybe, but I had the opposite result. I've seen people blame batch size when the real issue was much more mundane. Could be a workload difference, but I wouldn't generalize too fast from one setup. The logs were somehow calmer than I was.
Re: serving on Blackwell for FLUX - kernelmike - 08-31-2026
Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?
Re: serving on Blackwell for FLUX - vectorport - 09-01-2026
Need a little more detail here. What model, what runtime, and roughly what context size?
Re: serving on Blackwell for FLUX - sysstation69 - 09-02-2026
A couple things I'd check first. If you're on vLLM, I'd confirm the version first and then reduce variables until the behavior is obvious. Once you know the plain path works, then add the faster path back in piece by piece.
I'm happy to test one more variant if you want a second data point.
Re: serving on Blackwell for FLUX - edgecoast - 09-02-2026
My first guess would be this: On 4090, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: serving on Blackwell for FLUX - cacheline - 09-02-2026
I don't think that's the whole story. I've seen people blame long context when the real issue was much more mundane. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: serving on Blackwell for FLUX - circuitkite - 09-02-2026
Thanks, that helps. I don't think your conclusion is far off.
Re: serving on Blackwell for FLUX - oxidegrove40 - 09-03-2026
Quote:Thanks, that helps. I don't think your conclusion is far off.
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at router policy. Not glamorous advice, but repeatable baselines save a lot of time.