Posts: 59
Threads: 8
Joined: Sep 2026
Reputation:
0
Did a bit of testing and thought I'd post the result here.
I tried this with Mixtral using Ollama on M3 Max. The headline result was better than I expected, but prefill/decode split still ended up being the limiting factor.
Happy to post more details if anyone wants them.
If I had to summarize the lesson so far, it would be that clean setup and observability beat clever guesses.
Posts: 184
Threads: 23
Joined: Sep 2026
Reputation:
0
A couple things I'd check first. On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time. Turns out 'just one more test' was a lie.
Mostly CUDA, sometimes wrong, usually measured
Posts: 45
Threads: 6
Joined: Sep 2026
Reputation:
0
subscribed, curious where this lands.
Posts: 134
Threads: 18
Joined: Sep 2026
Reputation:
0
A couple questions before I guess too hard. Do you see the same thing with a smaller batch or shorter prompt?
watching release notes so you don't have to
Posts: 50
Threads: 3
Joined: Sep 2026
Reputation:
0
I tested this tonight. My result was mixed: first run looked bad, second run looked normal.
Posts: 40
Threads: 4
Joined: Sep 2026
Reputation:
0
Ran a quick check on my box. My result was mixed: first run looked bad, second run looked normal.
trying not to rebuild again
Posts: 40
Threads: 4
Joined: Sep 2026
Reputation:
0
same here on my side.
trying not to rebuild again
Posts: 50
Threads: 3
Joined: Sep 2026
Reputation:
0
that fixed it for me too.
Posts: 134
Threads: 18
Joined: Sep 2026
Reputation:
0
Ran a quick check on my box. The issue went away once I changed one setting and restarted the stack cleanly.
watching release notes so you don't have to
Posts: 40
Threads: 4
Joined: Sep 2026
Reputation:
0
A couple questions before I guess too hard. Is this local-only or are you routing anything externally?
trying not to rebuild again
