industry move that matters for local users

[deepseeked]

Interesting, I hadn't tried that. I don't think your conclusion is far off.

[dockriver]

Tried to reproduce it just now. On GB200 with MLX, I saw basically the same behavior. The surprise was that memory pressure showed up before throughput collapsed.

[audiopoint18]

same here on my side.

[southpatch]

What exact setup are you running? What model, what runtime, and roughly what context size?

[amberrunner]

Interesting, I hadn't tried that. I ended up in about the same place using LM Studio. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.

There are a few different conversations hiding inside this thread, so being explicit about the goal helps.

[deepseeked]

What worked for me was pretty simple. If you're on MLX, I'd confirm the version first and then reduce variables until the behavior is obvious.