+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Evaluation & Testing (https://www.datasculpting.ai/inference/forumdisplay.php?fid=16)
+--- Forum: Model Comparisons (https://www.datasculpting.ai/inference/forumdisplay.php?fid=67)
+--- Thread: flux pipeline local tips (/showthread.php?tid=1097)
Pages:
1
2
flux pipeline local tips - turboscope - 10-13-2026
I figured I'd ask here before changing everything.
I'm mostly working with FLUX in Ollama on Threadripper box. The part I'm trying to understand is long context.
Would appreciate practical advice rather than theory.
I care more about stability than squeezing the last possible tokens/sec out of it.
Re: flux pipeline local tips - cozykite - 10-13-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
I'm happy to test one more variant if you want a second data point.
Re: flux pipeline local tips - relaynest82 - 10-13-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at INT4. Once you know the plain path works, then add the faster path back in piece by piece.
Re: flux pipeline local tips - vectorshed43 - 10-14-2026
Not sure I agree there. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Worth testing the simpler explanation before adopting the more interesting one.
Re: flux pipeline local tips - amberpatch25 - 10-14-2026
Good point. I ended up in about the same place using llama.cpp. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: flux pipeline local tips - circuitwire58 - 10-17-2026
What worked for me was pretty simple. If you're on PyTorch, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.
Re: flux pipeline local tips - southspark - 10-18-2026
that fixed it for me too.
Re: flux pipeline local tips - mintforge - 10-18-2026
Quote:that fixed it for me too.
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: flux pipeline local tips - plainember52 - 10-18-2026
That matches what I was seeing. I ended up in about the same place using LM Studio. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: flux pipeline local tips - mintstation99 - 10-18-2026
Quote:That matches what I was seeing. I ended up in about the same place using LM Studio. Worth rechecking your defaults though, because one bad assumption
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.