flux pipeline local tips on B200

[latentcraft]

A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at prefix caching. Once you know the plain path works, then add the faster path back in piece by piece.

I'm happy to test one more variant if you want a second data point.

[signalcraft14]

Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.

[northgarden37]

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at token cost. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[promptspark58]

Not sure I agree there. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Could be a workload difference, but I wouldn't generalize too fast from one setup.