+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Miscellaneous Discussion (https://www.datasculpting.ai/inference/forumdisplay.php?fid=24)
+--- Forum: General (https://www.datasculpting.ai/inference/forumdisplay.php?fid=25)
+--- Thread: small win thread for DeepSeek (/showthread.php?tid=705)
small win thread for DeepSeek - ggufwire22 - 10-03-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
Maybe useful as a starting point: SGLang on dual 3090, keeping KV cache pressure in mind from the start.
Maybe useful as a starting point for newer folks.
Re: small win thread for DeepSeek - plaingarden18 - 10-03-2026
Thanks, that helps. That matches what I saw on dual 3090.
Re: small win thread for DeepSeek - rocmrunner40 - 10-04-2026
Good point. That matches what I saw on 4090. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: small win thread for DeepSeek - torchstation - 10-04-2026
Good point. That matches what I saw on A6000. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps.
Re: small win thread for DeepSeek - tensorkite - 10-04-2026
That matches what I was seeing. I ended up in about the same place using SGLang.
Re: small win thread for DeepSeek - cacheplane98 - 10-05-2026
I tested this tonight. On Threadripper box with vLLM, I saw basically the same behavior.
Re: small win thread for DeepSeek - plaingarden18 - 10-05-2026
Need a little more detail here. What model, what runtime, and roughly what context size?
Re: small win thread for DeepSeek - marbletrack - 10-05-2026
What exact setup are you running? What model, what runtime, and roughly what context size?
Re: small win thread for DeepSeek - ggufwire22 - 10-08-2026
Interesting, I hadn't tried that. That matches what I saw on Threadripper box. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: small win thread for DeepSeek - marbletrack - 10-09-2026
subscribed, curious where this lands.
There are a few different conversations hiding inside this thread, so being explicit about the goal helps. Turns out 'just one more test' was a lie.