+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Generative AI (https://www.datasculpting.ai/inference/forumdisplay.php?fid=17)
+--- Forum: Video (https://www.datasculpting.ai/inference/forumdisplay.php?fid=70)
+--- Thread: multimodal local stack that feels real on M3 Max (/showthread.php?tid=653)
Pages:
1
2
multimodal local stack that feels real on M3 Max - paperrunner - 10-01-2026
Wanted to compare notes with people doing similar things.
My current setup is MLX with Threadripper box, and I keep running into MoE serving.
Would appreciate practical advice rather than theory.
Re: multimodal local stack that feels real on M3 Max - cachelab - 10-02-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: multimodal local stack that feels real on M3 Max - audiopath66 - 10-02-2026
Interesting, I hadn't tried that. The pattern around structured output has been pretty similar for me. I got better results once I stopped chasing one metric and just looked at whether the workflow felt usable.
Re: multimodal local stack that feels real on M3 Max - forgemixer - 10-02-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tokens/sec. Once you know the plain path works, then add the faster path back in piece by piece.
Re: multimodal local stack that feels real on M3 Max - audiopath66 - 10-05-2026
I'd approach it in a boring order. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at MoE serving. Once you know the plain path works, then add the faster path back in piece by piece.
Re: multimodal local stack that feels real on M3 Max - turboarch95 - 10-08-2026
Appreciate the detail. That matches what I saw on Threadripper box. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: multimodal local stack that feels real on M3 Max - sparrowworks88 - 10-09-2026
following this because I'm in the same boat.
Re: multimodal local stack that feels real on M3 Max - quietarch - 10-10-2026
That matches what I was seeing. That matches what I saw on GB200.
Re: multimodal local stack that feels real on M3 Max - latentpeak22 - 10-10-2026
Tried to reproduce it just now. On 7900 XTX with LM Studio, I saw basically the same behavior.
Re: multimodal local stack that feels real on M3 Max - visionforge - 10-11-2026
Tried to reproduce it just now. My result was mixed: first run looked bad, second run looked normal. The surprise was that memory pressure showed up before throughput collapsed.