+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Open Weight (https://www.datasculpting.ai/inference/forumdisplay.php?fid=49)
+---- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=51)
+---- Thread: recipes for a good local stack (/showthread.php?tid=133)
Pages:
1
2
recipes for a good local stack - audioriver - 09-04-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Just sharing a setup that ended up working reasonably well: Kimi, MLX, 7900 XTX, with a lot of attention on tool calling.
Maybe useful as a starting point for newer folks.
Re: recipes for a good local stack - vramstream - 09-06-2026
Can you fill in a bit more context? Are you testing this on GB200 or something else?
Re: recipes for a good local stack - marbletrack - 09-06-2026
What worked for me was pretty simple. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at prefix caching. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: recipes for a good local stack - rocmriley - 09-07-2026
I'd approach it in a boring order. On mini PC, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: recipes for a good local stack - audionode8 - 09-07-2026
Not sure I agree there. The reported numbers can look compelling, but in ordinary usage I didn't see the same benefit.
Re: recipes for a good local stack - rocmriley - 09-07-2026
same here on my side.
Re: recipes for a good local stack - deltagrid - 09-08-2026
I'd start with the simpler stack.
Re: recipes for a good local stack - vectorport - 09-11-2026
A couple things I'd check first. On B200, the easiest win is often to back off the aggressive settings and see whether the problem survives. Not glamorous advice, but repeatable baselines save a lot of time.
Re: recipes for a good local stack - nanopatch - 09-11-2026
A couple things I'd check first. If you're on TensorRT-LLM, I'd confirm the version first and then reduce variables until the behavior is obvious.
Re: recipes for a good local stack - audioriver - 09-11-2026
A couple things I'd check first. If you're on PyTorch, I'd confirm the version first and then reduce variables until the behavior is obvious. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.