flux pipeline local tips for Phi

[kernelrunner]

This keeps coming up in conversation so I figured I'd start a thread.

One thing I keep noticing is that people underestimate consumer GPUs until they actually try to run this stuff for more than one afternoon.

Curious whether that lines up with what other people are seeing.

[modrunner27]

I think the bottleneck is somewhere else. I wouldn't optimize around that first. On my side the bigger swing came from the runtime and not the model. Could be a workload difference, but I wouldn't generalize too fast from one setup.

I'm happy to test one more variant if you want a second data point.

[rocmpath]

Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly. The main thing that moved was latency, not quality.