+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: LLM Models (https://www.datasculpting.ai/inference/forumdisplay.php?fid=12)
+--- Forum: Open Weight (https://www.datasculpting.ai/inference/forumdisplay.php?fid=49)
+---- Forum: Local (https://www.datasculpting.ai/inference/forumdisplay.php?fid=51)
+---- Thread: recipes for a good local stack (/showthread.php?tid=269)
Pages:
1
2
Re: recipes for a good local stack - llmpeak - 09-19-2026
Did a small comparison this afternoon. On 5090 with SGLang, I saw basically the same behavior.
Re: recipes for a good local stack - audionode8 - 09-21-2026
A couple questions before I guess too hard. What model, what runtime, and roughly what context size?
Re: recipes for a good local stack - cachebench - 09-24-2026
Good point. That matches what I saw on 7900 XTX. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: recipes for a good local stack - thetagrid15 - 09-24-2026
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher. The main thing that moved was latency, not quality.
Re: recipes for a good local stack - marbleyard32 - 09-24-2026
Appreciate the detail. I ended up in about the same place using vLLM. The biggest change for me was simplifying the stack instead of adding one more optimization layer.