10-01-2026, 03:51 AM
Thought I'd post a small recipe because a lot of threads ask similar things.
Just sharing a setup that ended up working reasonably well: DeepSeek, SGLang, M3 Max, with a lot of attention on batch size.
Feel free to tear it apart if you see something dumb.
This is mostly for local work rather than public serving.
Just sharing a setup that ended up working reasonably well: DeepSeek, SGLang, M3 Max, with a lot of attention on batch size.
Feel free to tear it apart if you see something dumb.
This is mostly for local work rather than public serving.
routing beats arguing over one perfect model