+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Open Weight (https://www.datasculpting.ai/inference/forumdisplay.php?fid=49)
+---- Forum: Hosted (https://www.datasculpting.ai/inference/forumdisplay.php?fid=50)
+---- Thread: best setup on 24GB VRAM on M3 Max (/showthread.php?tid=310)
best setup on 24GB VRAM on M3 Max - torchsmith25 - 09-17-2026
Spent some time on this today and wanted to compare notes.
I'm comparing two approaches here: keep it simple on 5090, or squeeze more out of the stack and deal with the complexity.
Trying to avoid rebuilding the whole thing twice.
Re: best setup on 24GB VRAM on M3 Max - southpatch - 09-18-2026
Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal. I expected the model choice to matter more than it did.
Re: best setup on 24GB VRAM on M3 Max - amberplane - 09-19-2026
Quote:Did a small comparison this afternoon. My result was mixed: first run looked bad, second run looked normal. I expected the model choice to matter more
Not sure I agree there. I've seen people blame tokens/sec when the real issue was much more mundane.
Re: best setup on 24GB VRAM on M3 Max - emberrunner - 09-19-2026
following this because I'm in the same boat.
Re: best setup on 24GB VRAM on M3 Max - cozyarch - 09-19-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: best setup on 24GB VRAM on M3 Max - latentcraft - 09-22-2026
A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. Not glamorous advice, but repeatable baselines save a lot of time.
Re: best setup on 24GB VRAM on M3 Max - cozyarch - 09-22-2026
Quote:A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. Not glamorous advice, but r
Maybe, but I had the opposite result. I've seen people blame consumer GPUs when the real issue was much more mundane. Could be a workload difference, but I wouldn't generalize too fast from one setup.
Re: best setup on 24GB VRAM on M3 Max - kernelmike - 09-22-2026
Quote:[quote]A couple things I'd check first. Before rebuilding anything, I'd test one clean baseline with FLUX and compare from there. Not glamorous advice
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at tool calling. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.