flux pipeline local tips on Threadripper box

[vramforge]

Posting this in case it saves someone else a little time.

I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about NVFP4.

Simple and boring is a valid answer.

My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.

[homelabhenry]

That matches what I was seeing. The pattern around router policy has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable. I keep promising myself this is the last rebuild.

[rocmstack25]

I tested this tonight. On 7900 XTX with PyTorch, I saw basically the same behavior.

[routeforge64]

Thanks, that helps. I don't think your conclusion is far off.

[quantsupply]

Good point. That matches what I saw on B200.

[mangopath4]

What worked for me was pretty simple. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.

[fusedtinker]

Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.

[signalcraft14]

worth checking the driver first.

[plainline]

I tested this tonight. My result was mixed: first run looked bad, second run looked normal. I expected the model choice to matter more than it did.

[macrostack69]

I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at small local stacks.

What exact setup are you running? Is this local-only or are you routing anything externally?