tools worth paying for with MLX

[batchshed]

Thought I'd post a small recipe because a lot of threads ask similar things.

Just sharing a setup that ended up working reasonably well: DeepSeek, SGLang, 5090, with a lot of attention on structured output.

Maybe useful as a starting point for newer folks. At this point the real bottleneck is my patience.

[bytelab]

That matches what I was seeing. The pattern around tokens/sec has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[modrunner27]

That matches what I was seeing. The pattern around tokens/sec has been pretty similar for me. The biggest change for me was simplifying the stack inst

Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed.

[mangopeak72]

I don't think that's the whole story. I've seen people blame KV cache pressure when the real issue was much more mundane.

[tinycontext]

I don't think that's the whole story. I've seen people blame KV cache pressure when the real issue was much more mundane.

I hit almost the same issue last night.

[vramarch]

Good point. I don't think your conclusion is far off. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

[mangopeak72]

following this because I'm in the same boat.

[dockworks23]

worth checking the driver first.

[deepseeked]

worth checking the driver first.

Need a little more detail here. Is this local-only or are you routing anything externally?