+- Inference (https://www.datasculpting.ai/inference)
+-- Forum: Evaluation & Testing (https://www.datasculpting.ai/inference/forumdisplay.php?fid=16)
+--- Forum: Benchmarks (https://www.datasculpting.ai/inference/forumdisplay.php?fid=65)
+--- Thread: comparing models without fooling yourself on GB200 (/showthread.php?tid=1470)
Pages:
1
2
Re: comparing models without fooling yourself on GB200 - deltastation - 10-18-2026
I tested this tonight. The issue went away once I changed one setting and restarted the stack cleanly.
comparing models without fooling yourself on GB200 - griffinloom66 - 10-18-2026
Thought I'd post a small recipe because a lot of threads ask similar things.
This isn't revolutionary, but the combination of vLLM and dual 3090 felt surprisingly usable once I tuned around tool calling.
Feel free to tear it apart if you see something dumb.
My use case is pretty ordinary: local chat, some document work, and occasional batch jobs.
Re: comparing models without fooling yourself on GB200 - llmtrack26 - 10-19-2026
What worked for me was pretty simple. On A6000, the easiest win is often to back off the aggressive settings and see whether the problem survives. Once you know the plain path works, then add the faster path back in piece by piece.
Re: comparing models without fooling yourself on GB200 - kernelghost - 10-19-2026
I tested this tonight. I couldn't reproduce the slowdown until I pushed context much higher. I expected the model choice to matter more than it did.
Re: comparing models without fooling yourself on GB200 - plainember52 - 10-19-2026
Interesting, I hadn't tried that. I ended up in about the same place using SGLang. Worth rechecking your defaults though, because one bad assumption can make the rest of the thread look confusing.
Re: comparing models without fooling yourself on GB200 - vramridge - 10-19-2026
I'd start with the simpler stack.
Re: comparing models without fooling yourself on GB200 - quiethouse - 10-19-2026
A couple things I'd check first. Start by verifying the driver/runtime combo, then test with a smaller prompt, then look at continuous batching. A lot of these threads end up being one bad config plus a pile of conclusions drawn from it.
Re: comparing models without fooling yourself on GB200 - circuitworks90 - 10-19-2026
same here on my side.
Re: comparing models without fooling yourself on GB200 - batchpeak - 10-19-2026
That matches what I was seeing. I ended up in about the same place using SGLang. The biggest change for me was simplifying the stack instead of adding one more optimization layer.
Re: comparing models without fooling yourself on GB200 - deltastation - 10-19-2026
I'd approach it in a boring order. If you're on llama.cpp, I'd confirm the version first and then reduce variables until the behavior is obvious.