+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Drivers / Frameworks / Toolkits (https://www.datasculpting.ai/inference/forumdisplay.php?fid=8)
+--- Forum: Apple / MLX (https://www.datasculpting.ai/inference/forumdisplay.php?fid=33)
+--- Thread: Apple unified memory limits: setup advice (/showthread.php?tid=1022)
Pages:
1
2
Re: Apple unified memory limits: setup advice - orbitline - 10-14-2026
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
Re: Apple unified memory limits: setup advice - mellowstream - 10-14-2026
Quote:I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there. Not glamorous advice,
Can you fill in a bit more context? Is this local-only or are you routing anything externally?
Re: Apple unified memory limits: setup advice - cudacrate - 10-14-2026
Quote:[quote]I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Mixtral and compare from there. Not glamorous a
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with Llama and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: Apple unified memory limits: setup advice - torchshed - 10-14-2026
A couple questions before I guess too hard. Are you testing this on GB200 or something else?
Re: Apple unified memory limits: setup advice - dockkite69 - 10-16-2026
That matches what I was seeing. The pattern around continuous batching has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: Apple unified memory limits: setup advice - mellowstream - 10-16-2026
My first guess would be this: Before rebuilding anything, I'd test one clean baseline with Mistral and compare from there. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
I'm happy to test one more variant if you want a second data point.