+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Optimization & Performance (https://www.datasculpting.ai/inference/forumdisplay.php?fid=10)
+--- Forum: Benchmarks (https://www.datasculpting.ai/inference/forumdisplay.php?fid=43)
+--- Thread: domain tune without wrecking base behavior (/showthread.php?tid=71)
Pages:
1
2
domain tune without wrecking base behavior - routearch - 08-30-2026
Did a bit of testing and thought I'd post the result here.
I tried this with DeepSeek using vLLM on M3 Max. The headline result was better than I expected, but tokens/sec still ended up being the limiting factor.
Curious if anyone else landed in the same place.
Re: domain tune without wrecking base behavior - kernelarch - 09-01-2026
My first guess would be this: On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Re: domain tune without wrecking base behavior - quiethouse - 09-01-2026
Quote:My first guess would be this: On 5090, the easiest win is often to back off the aggressive settings and see whether the problem survives.
Ran a quick check on my box. On 4090 with llama.cpp, I saw basically the same behavior. I expected the model choice to matter more than it did.
Re: domain tune without wrecking base behavior - deltastation - 09-01-2026
Ran a quick check on my box. I couldn't reproduce the slowdown until I pushed context much higher.
Re: domain tune without wrecking base behavior - batchfield55 - 09-02-2026
I don't think that's the whole story. I get why that seems right, but on GB200 the bottleneck felt elsewhere. Could be a workload difference, but I wouldn't generalize too fast from one setup.
If you post your exact settings, people here can probably narrow it down pretty fast.
Re: domain tune without wrecking base behavior - signalcraft14 - 09-02-2026
that fixed it for me too.
Re: domain tune without wrecking base behavior - quiethouse - 09-02-2026
Not sure I agree there. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock that in as the lesson yet.
Re: domain tune without wrecking base behavior - localtinker - 09-02-2026
Quote:Not sure I agree there. I get why that seems right, but on dual 3090 the bottleneck felt elsewhere. Not saying you're wrong, just that I wouldn't lock
Thanks, that helps. I ended up in about the same place using SGLang. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: domain tune without wrecking base behavior - routearch - 09-04-2026
What worked for me was pretty simple. Before rebuilding anything, I'd test one clean baseline with Phi and compare from there. Once you know the plain path works, then add the faster path back in piece by piece.
Re: domain tune without wrecking base behavior - cacheline - 09-04-2026
What exact setup are you running? Are you testing this on B200 or something else?