when a plain Python stack is enough with PyTorch

[ggufwire22]

What exact setup are you running? Do you see the same thing with a smaller batch or shorter prompt?

[quietcraft]

Not sure I agree there. I've seen people blame router policy when the real issue was much more mundane. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.

[cobaltlab]

Maybe, but I had the opposite result. I get why that seems right, but on 5090 the bottleneck felt elsewhere.

[macrolab]

Maybe, but I had the opposite result. I get why that seems right, but on 5090 the bottleneck felt elsewhere.

That matches what I was seeing. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[phasenode]

I hit almost the same issue last night.