10-07-2026, 03:16 AM
I'm trying to sanity-check my setup.
I'm mostly working with Mistral in MLX on 5090. The part I'm trying to understand is continuous batching.
Not looking for the perfect answer, mostly what worked in real use.
I care more about stability than squeezing the last possible tokens/sec out of it.
I'm mostly working with Mistral in MLX on 5090. The part I'm trying to understand is continuous batching.
Not looking for the perfect answer, mostly what worked in real use.
I care more about stability than squeezing the last possible tokens/sec out of it.