what music do you benchmark to for Mixtral

[thetatinker77]

Posting this in case it saves someone else a little time.

I'm not trying to build a perfect architecture diagram. I just want a sane starting point around LM Studio and M3 Max.

I'm especially interested in mistakes worth avoiding.

[audiorunner74]

Posting this in case it saves someone else a little time.

I think the bottleneck is somewhere else. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit. Worth testing the simpler explanation before adopting the more interesting one.

[deltapoint17]

I hit almost the same issue last night.

[kernelghost]

same here on my side.

[amberpatch25]

I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.

[phasetrack]

Good point. I ended up in about the same place using llama.cpp. The biggest change for me was simplifying the stack instead of adding one more optimization layer.