+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: NVIDIA / CUDA (https://www.datasculpting.ai/inference/forumdisplay.php?fid=31)
+--- Thread: Blackwell deployment chatter for FLUX (/showthread.php?tid=622)
Pages:
1
2
Blackwell deployment chatter for FLUX - thetapath - 10-01-2026
Curious what people here are seeing.
I'm mostly working with Qwen in LM Studio on 4090. The part I'm trying to understand is prefix caching.
If you've tested something similar, what did you keep and what did you stop bothering with?
Re: Blackwell deployment chatter for FLUX - edgecoast - 10-01-2026
What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?
Re: Blackwell deployment chatter for FLUX - kernelmike - 10-01-2026
Good point. The pattern around MoE serving has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: Blackwell deployment chatter for FLUX - dockember - 10-01-2026
Quote:Good point. The pattern around MoE serving has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make
I'd start with the simpler stack.
Re: Blackwell deployment chatter for FLUX - audiopath66 - 10-01-2026
Good point. The pattern around batch size has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: Blackwell deployment chatter for FLUX - sparrowkite53 - 10-04-2026
Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
Re: Blackwell deployment chatter for FLUX - cobaltspark - 10-04-2026
Quote:Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model.
I don't think that's the whole story. I've seen people blame tokens/sec when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: Blackwell deployment chatter for FLUX - phaseport36 - 10-04-2026
I tested this tonight. On Threadripper box with TensorRT-LLM, I saw basically the same behavior. The main thing that moved was latency, not quality.
I'm happy to test one more variant if you want a second data point.
Re: Blackwell deployment chatter for FLUX - sparrowkite53 - 10-05-2026
Not sure I agree there. I've seen people blame consumer GPUs when the real issue was much more mundane.
Re: Blackwell deployment chatter for FLUX - phaseport36 - 10-05-2026
worth checking the driver first.