10-16-2026, 07:05 AM
Posting this in case it saves someone else a little time.
I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about MoE serving.
I'm especially interested in mistakes worth avoiding.
This is mostly for local work rather than public serving.
I'm helping a small team set up a local workflow. We don't need a giant stack, but we do care about MoE serving.
I'm especially interested in mistakes worth avoiding.
This is mostly for local work rather than public serving.
throughput matters, but tail latency matters too