09-28-2026, 09:32 AM
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of PyTorch and 4090 felt surprisingly usable once I tuned around MoE serving.
Feel free to tear it apart if you see something dumb.
This is mostly for local work rather than public serving.
This isn't revolutionary, but the combination of PyTorch and 4090 felt surprisingly usable once I tuned around MoE serving.
Feel free to tear it apart if you see something dumb.
This is mostly for local work rather than public serving.
7900 XTX / ROCm / somehow still optimistic