10-11-2026, 10:54 AM
Spent some time on this today and wanted to compare notes.
I've gone back and forth between PyTorch and SGLang for Phi. The interesting difference for me has been INT4.
What would you pick if you had to live with it for a few months?
This is mostly for local work rather than public serving.
I've gone back and forth between PyTorch and SGLang for Phi. The interesting difference for me has been INT4.
What would you pick if you had to live with it for a few months?
This is mostly for local work rather than public serving.
7900 XTX / ROCm / somehow still optimistic