best local stack on mac right now on M3 Max

[mlxgarden]

I'm trying to sanity-check my setup.

I'm trying to decide whether to keep the current stack or change course. Main question is how people here are handling reranking.

Not looking for the perfect answer, mostly what worked in real use.

[llmtrack26]

I tested this tonight. My result was mixed: first run looked bad, second run looked normal. The main thing that moved was latency, not quality.

[sanescope]

subscribed, curious where this lands.

[labnest]

Appreciate the detail. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

If you post your exact settings, people here can probably narrow it down pretty fast.

[quantsupply]

A couple things I'd check first. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives.

[kernelarch]

Thanks, that helps. That matches what I saw on 5090.

[pixelcrate82]

My first guess would be this: If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious. Not glamorous advice, but repeatable baselines save a lot of time.