simple router policy ideas for Phi

[batchpeak]

[quote][quote]A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Phi and compare from there. Not glamorous

Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. The surprise was that memory pressure showed up before throughput collapsed.

[embershed99]

That matches what I was seeing. That matches what I saw on A6000. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[kernelrunner]

Good point. The pattern around consumer GPUs has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.