prefill/decode separation in practice with PyTorch

[routearch]

I don't think that's the whole story. I've seen people blame VRAM headroom when the real issue was much more mundane. Worth testing the simpler explanation before adopting the more interesting one. Turns out 'just one more test' was a lie.