private docs and local pipelines on M3 Max

[batchwire]

Maybe I'm missing something obvious.

I'm mostly working with Kimi in LM Studio on M3 Max. The part I'm trying to understand is workstation thermals.

Not looking for the perfect answer, mostly what worked in real use.

[rocmriley]

That matches what I was seeing. The pattern around long context has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[batchshed]

A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious.

[mintstation99]

Good point. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.

[mlxgarden]

following this because I'm in the same boat.

[tensorkite]

A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at prefix caching.

[mintstation99]

Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.

[audionode8]

Appreciate the detail. I ended up in about the same place using PyTorch.

[cachebench]

Appreciate the detail. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing. The logs were somehow calmer than I was.

[rasterbench]

I think the bottleneck is somewhere else. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.