M3 Max for local use

[cudacrate]

Curious what people here are seeing.

I'm mostly working with Mixtral in SGLang on A6000. The part I'm trying to understand is KV cache pressure.

Would appreciate practical advice rather than theory.

This is mostly for local work rather than public serving.

[mellowstream]

I'd start with the simpler stack.

I'm happy to test one more variant if you want a second data point.