prefill/decode separation in practice with Ollama

[promptpoint]

I'd approach it in a boring order. On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.

[turboarch95]

I'd approach it in a boring order. On M3 Max, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not g

Appreciate the detail. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.

[batchcraft]

Need a little more detail here. Are you testing this on mini PC or something else?

[nanogarden]

I tested this tonight. My result was mixed: first run looked bad, second run looked normal.