+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Generative AI (https://www.datasculpting.ai/inference/forumdisplay.php?fid=17)
+--- Forum: Video (https://www.datasculpting.ai/inference/forumdisplay.php?fid=70)
+--- Thread: multimodal local stack that feels real: what are you seeing? (/showthread.php?tid=1067)
multimodal local stack that feels real: what are you seeing? - latentpeak22 - 10-11-2026
Spent some time on this today and wanted to compare notes.
I'm comparing two approaches here: keep it simple on M3 Max, or squeeze more out of the stack and deal with the complexity.
Interested in tradeoffs more than fan loyalty.
Re: multimodal local stack that feels real: what are you seeing? - thetaember38 - 10-11-2026
Tried to reproduce it just now. On A6000 with llama.cpp, I saw basically the same behavior. The logs were somehow calmer than I was.
Re: multimodal local stack that feels real: what are you seeing? - quietarch - 10-12-2026
Thanks, that helps. The pattern around tokens/sec has been pretty similar for me.
Re: multimodal local stack that feels real: what are you seeing? - visionforge - 10-12-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: multimodal local stack that feels real: what are you seeing? - southtinker - 10-13-2026
Quote:What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Kimi and compare from there. Once you know the plai
Interesting, I hadn't tried that. The pattern around router policy has been pretty similar for me. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: multimodal local stack that feels real: what are you seeing? - visionforge - 10-15-2026
My first guess would be this: Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at batch size. Not glamorous advice, but repeatable baselines save a lot of time.