+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Security, Privacy & Safety (https://www.datasculpting.ai/inference/forumdisplay.php?fid=23)
+--- Forum: Local Privacy (https://www.datasculpting.ai/inference/forumdisplay.php?fid=76)
+--- Thread: private docs and local pipelines on M3 Max (/showthread.php?tid=230)
Pages:
1
2
private docs and local pipelines on M3 Max - batchwire - 09-11-2026
Maybe I'm missing something obvious.
I'm mostly working with Kimi in LM Studio on M3 Max. The part I'm trying to understand is workstation thermals.
Not looking for the perfect answer, mostly what worked in real use.
Re: private docs and local pipelines on M3 Max - rocmriley - 09-12-2026
That matches what I was seeing. The pattern around long context has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: private docs and local pipelines on M3 Max - batchshed - 09-12-2026
A couple things I'd check first. If you're on LM Studio, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: private docs and local pipelines on M3 Max - mintstation99 - 09-12-2026
Good point. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: private docs and local pipelines on M3 Max - mlxgarden - 09-13-2026
following this because I'm in the same boat.
Re: private docs and local pipelines on M3 Max - tensorkite - 09-13-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at prefix caching.
Re: private docs and local pipelines on M3 Max - mintstation99 - 09-16-2026
Tried to reproduce it just now. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.
Re: private docs and local pipelines on M3 Max - audionode8 - 09-16-2026
Appreciate the detail. I ended up in about the same place using PyTorch.
Re: private docs and local pipelines on M3 Max - cachebench - 09-18-2026
Appreciate the detail. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing. The logs were somehow calmer than I was.
Re: private docs and local pipelines on M3 Max - rasterbench - 09-18-2026
I think the bottleneck is somewhere else. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.