09-27-2026, 04:27 AM
Wanted to compare notes with people doing similar things.
I'm mostly working with Mixtral in MLX on 5090. The part I'm trying to understand is tokens/sec.
If you've tested something similar, what did you keep and what did you stop bothering with?
This is mostly for local work rather than public serving.
I'm mostly working with Mixtral in MLX on 5090. The part I'm trying to understand is tokens/sec.
If you've tested something similar, what did you keep and what did you stop bothering with?
This is mostly for local work rather than public serving.
Mostly CUDA, sometimes wrong, usually measured