+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Hardware (https://www.datasculpting.ai/inference/forumdisplay.php?fid=4)
+--- Forum: GPUs (https://www.datasculpting.ai/inference/forumdisplay.php?fid=28)
+--- Thread: RTX 5090 local inference for FLUX (/showthread.php?tid=1379)
Pages:
1
2
RTX 5090 local inference for FLUX - flintriver50 - 10-17-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of llama.cpp and 7900 XTX felt surprisingly usable once I tuned around token cost.
Maybe useful as a starting point for newer folks.
Re: RTX 5090 local inference for FLUX - turbogrid11 - 10-18-2026
Quote:Thought I'd post a small recipe because a lot of threads ask similar things.
My first guess would be this: If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: RTX 5090 local inference for FLUX - rocmriley - 10-18-2026
A couple things I'd check first. If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: RTX 5090 local inference for FLUX - torchshed - 10-18-2026
Tried to reproduce it just now. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.
Re: RTX 5090 local inference for FLUX - rocmriley - 10-18-2026
A couple things I'd check first. If you're on Ollama, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
Re: RTX 5090 local inference for FLUX - signaltrack44 - 10-18-2026
Interesting, I hadn't tried that. The pattern around batch size has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: RTX 5090 local inference for FLUX - turbogrid11 - 10-18-2026
Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly.
Re: RTX 5090 local inference for FLUX - fusedhall - 10-19-2026
What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?
Re: RTX 5090 local inference for FLUX - vrambandit - 10-19-2026
What exact setup are you running? Is this local-only or are you routing anything externally?
Re: RTX 5090 local inference for FLUX - cudapoint - 10-19-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tokens/sec. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.