+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Open Weight (https://www.datasculpting.ai/inference/forumdisplay.php?fid=49)
+---- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=51)
+---- Thread: recipes for a good local stack on 4090 (/showthread.php?tid=106)
recipes for a good local stack on 4090 - mintstation99 - 09-02-2026
Did a bit of testing and thought I'd post the result here.
I tried this with Gemma using vLLM on dual 3090. The headline result was better than I expected, but quant drift still ended up being the limiting factor.
Nothing scientific here, just practical notes from a real setup.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Re: recipes for a good local stack on 4090 - visionforge - 09-02-2026
Can you fill in a bit more context? What model, what runtime, and roughly what context size?
Re: recipes for a good local stack on 4090 - vramstream - 09-03-2026
Interesting, I hadn't tried that. I ended up in about the same place using MLX.
Re: recipes for a good local stack on 4090 - mlxgarden - 09-03-2026
That matches what I was seeing. That matches what I saw on 7900 XTX.
Re: recipes for a good local stack on 4090 - vramstream - 09-03-2026
Quote:That matches what I was seeing. That matches what I saw on 7900 XTX.
I'd approach it in a boring order. Before rebuilding anything, I'd test one clean baseline with Qwen and compare from there.
Re: recipes for a good local stack on 4090 - relayforge - 09-04-2026
Need a little more detail here. Do you see the same thing with a smaller batch or shorter prompt?
Re: recipes for a good local stack on 4090 - mellowwire98 - 09-05-2026
I'd start with the simpler stack.