benchmarks that map to real use on M3 Max

[torchtrack24]

Posting this in case it saves someone else a little time.

If you were starting from scratch today for this use case, what would you choose first and what would you ignore?

Simple and boring is a valid answer.

[mellowsupply]

that fixed it for me too.

[audiopoint18]

Can you fill in a bit more context? Is this local-only or are you routing anything externally?

[quietcraft]

subscribed, curious where this lands.

[batchsize]

My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tool calling. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[quiettoken]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.

[amberkite]

What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[rocmriley]

What worked for me was pretty simple. On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.

[mlxgarden]

worth checking the driver first.

[mintstation99]

Did a small comparison this afternoon. On Threadripper box with Ollama, I saw basically the same behavior. I expected the model choice to matter more

I tested this tonight. On GB200 with MLX, I saw basically the same behavior. The main thing that moved was latency, not quality.