+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Performance Tuning (https://www.datasculpting.ai/inference/forumdisplay.php?fid=42)
+--- Thread: disaggregated inference thoughts on M3 Max (/showthread.php?tid=11)
Pages:
1
2
disaggregated inference thoughts on M3 Max - cozyarch - 08-22-2026
Has anyone else run into this?
My current setup is SGLang with 4090, and I keep running into prefix caching.
If you've tested something similar, what did you keep and what did you stop bothering with?
Re: disaggregated inference thoughts on M3 Max - edgeriver - 08-22-2026
Appreciate the detail. I don't think your conclusion is far off. The biggest change for me was simplifying the stack instead of adding one more optimization layer. Turns out 'just one more test' was a lie.
Re: disaggregated inference thoughts on M3 Max - tensorfield88 - 08-23-2026
That matches what I was seeing. That matches what I saw on 5090.
Re: disaggregated inference thoughts on M3 Max - labline - 08-23-2026
Tried to reproduce it just now. On A6000 with LM Studio, I saw basically the same behavior. I expected the model choice to matter more than it did.
Re: disaggregated inference thoughts on M3 Max - kernelmike - 08-24-2026
Maybe, but I had the opposite result. I get why that seems right, but on M3 Max the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: disaggregated inference thoughts on M3 Max - cudapeak - 08-24-2026
Tried to reproduce it just now. On GB200 with Ollama, I saw basically the same behavior.
Re: disaggregated inference thoughts on M3 Max - cozygrid12 - 08-24-2026
that fixed it for me too.
Re: disaggregated inference thoughts on M3 Max - kernelmike - 08-27-2026
Quote:that fixed it for me too.
Interesting, I hadn't tried that. The pattern around MoE serving has been pretty similar for me. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: disaggregated inference thoughts on M3 Max - rocmhouse90 - 08-27-2026
Appreciate the detail. I don't think your conclusion is far off. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: disaggregated inference thoughts on M3 Max - cudapeak - 08-30-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there.