08-29-2026, 11:59 PM
I figured I'd ask here before changing everything.
I'm mostly working with Mistral in SGLang on B200. The part I'm trying to understand is INT4.
Not looking for the perfect answer, mostly what worked in real use.
This is mostly for local work rather than public serving.
I expected the biggest difference to come from the model, but the stack around it mattered at least as much.
I'm mostly working with Mistral in SGLang on B200. The part I'm trying to understand is INT4.
Not looking for the perfect answer, mostly what worked in real use.
This is mostly for local work rather than public serving.
I expected the biggest difference to come from the model, but the stack around it mattered at least as much.
Apple Silicon • MLX mostly